-- Logs begin at Thu 2022-07-21 15:19:21 UTC, end at Thu 2022-07-21 16:53:16 UTC. -- Jul 21 15:49:45.419218 dv-cinder-850668-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 21 15:49:45.430297 dv-cinder-850668-1-iscsi devstack@keystone.service[80228]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 21 15:49:45.432365 dv-cinder-850668-1-iscsi devstack@keystone.service[80228]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 21 15:49:45 2022] *** Jul 21 15:49:45.432365 dv-cinder-850668-1-iscsi devstack@keystone.service[80228]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 21 15:49:45.432365 dv-cinder-850668-1-iscsi devstack@keystone.service[80228]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 21 15:49:45.432365 dv-cinder-850668-1-iscsi devstack@keystone.service[80228]: nodename: dv-cinder-850668-1-iscsi Jul 21 15:49:45.432365 dv-cinder-850668-1-iscsi devstack@keystone.service[80228]: machine: x86_64 Jul 21 15:49:45.432365 dv-cinder-850668-1-iscsi devstack@keystone.service[80228]: clock source: unix Jul 21 15:49:45.432365 dv-cinder-850668-1-iscsi devstack@keystone.service[80228]: pcre jit disabled Jul 21 15:49:45.432365 dv-cinder-850668-1-iscsi devstack@keystone.service[80228]: detected number of CPU cores: 4 Jul 21 15:49:45.432365 dv-cinder-850668-1-iscsi devstack@keystone.service[80228]: current working directory: / Jul 21 15:49:45.432365 dv-cinder-850668-1-iscsi devstack@keystone.service[80228]: detected binary path: /usr/bin/uwsgi-core Jul 21 15:49:45.432365 dv-cinder-850668-1-iscsi devstack@keystone.service[80228]: your processes number limit is 47680 Jul 21 15:49:45.432963 dv-cinder-850668-1-iscsi devstack@keystone.service[80228]: your memory page size is 4096 bytes Jul 21 15:49:45.432963 dv-cinder-850668-1-iscsi devstack@keystone.service[80228]: detected max file descriptor number: 2048 Jul 21 15:49:45.432963 dv-cinder-850668-1-iscsi devstack@keystone.service[80228]: lock engine: pthread robust mutexes Jul 21 15:49:45.432963 dv-cinder-850668-1-iscsi devstack@keystone.service[80228]: thunder lock: enabled Jul 21 15:49:45.432963 dv-cinder-850668-1-iscsi devstack@keystone.service[80228]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 21 15:49:45.432963 dv-cinder-850668-1-iscsi devstack@keystone.service[80228]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 21 15:49:45.476604 dv-cinder-850668-1-iscsi devstack@keystone.service[80228]: Python main interpreter initialized at 0x55cacbf019c0 Jul 21 15:49:45.476604 dv-cinder-850668-1-iscsi devstack@keystone.service[80228]: python threads support enabled Jul 21 15:49:45.476604 dv-cinder-850668-1-iscsi devstack@keystone.service[80228]: your server socket listen backlog is limited to 100 connections Jul 21 15:49:45.476889 dv-cinder-850668-1-iscsi devstack@keystone.service[80228]: your mercy for graceful operations on workers is 90 seconds Jul 21 15:49:45.476975 dv-cinder-850668-1-iscsi devstack@keystone.service[80228]: mapped 403077 bytes (393 KB) for 2 cores Jul 21 15:49:45.477050 dv-cinder-850668-1-iscsi devstack@keystone.service[80228]: *** Operational MODE: preforking *** Jul 21 15:49:45.477320 dv-cinder-850668-1-iscsi devstack@keystone.service[80228]: *** uWSGI is running in multiple interpreter mode *** Jul 21 15:49:45.477320 dv-cinder-850668-1-iscsi devstack@keystone.service[80228]: spawned uWSGI master process (pid: 80228) Jul 21 15:49:45.477527 dv-cinder-850668-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 21 15:49:45.479431 dv-cinder-850668-1-iscsi devstack@keystone.service[80228]: spawned uWSGI worker 1 (pid: 80237, cores: 1) Jul 21 15:49:45.479431 dv-cinder-850668-1-iscsi devstack@keystone.service[80228]: spawned uWSGI worker 2 (pid: 80238, cores: 1) Jul 21 15:49:45.479431 dv-cinder-850668-1-iscsi devstack@keystone.service[80228]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 21 15:49:50.939346 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 15:49:50.939346 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: from cryptography.utils import int_from_bytes Jul 21 15:49:50.940148 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 15:49:50.940148 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: from cryptography.utils import int_from_bytes Jul 21 15:49:50.940265 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 15:49:50.940265 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: from cryptography.utils import int_from_bytes Jul 21 15:49:50.940678 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 15:49:50.940678 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: from cryptography.utils import int_from_bytes Jul 21 15:49:50.990358 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 21 15:49:50.990635 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 21 15:49:50.990776 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] command line args: [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 21 15:49:50.990927 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 21 15:49:50.991068 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 21 15:49:50.991203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 21 15:49:50.991342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ================================================================================ {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 21 15:49:50.991470 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] command line args: [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 21 15:49:50.991596 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] admin_token = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.991732 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 21 15:49:50.991868 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] config_dir = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.992014 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ================================================================================ {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 21 15:49:50.992140 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.992273 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] admin_token = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.992365 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] config_source = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.992469 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] config_dir = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.992592 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] debug = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.992808 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.993002 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.993355 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] config_source = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.993355 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] debug = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.993641 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.993765 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.993907 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.994032 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.994032 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.994259 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.994428 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.994611 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.994814 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.994998 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.995141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.995289 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_config_append = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.995419 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] log_config_append = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.995543 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.995667 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.995799 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_dir = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.995968 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] log_dir = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.996107 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_file = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.996187 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] log_file = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.996306 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.996390 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.996483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.996606 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.996753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.996883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.997010 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.997147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.997278 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.997408 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.997548 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.997735 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.997865 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.998003 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.998115 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.998203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.998279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.998405 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.998483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.998556 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.998556 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.998556 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.998864 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.998941 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.999021 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.999154 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.999234 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.999361 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.999447 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.999447 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.999588 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.999588 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] publish_errors = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.999731 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.999811 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.999885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] publish_errors = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:50.999962 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:51.000036 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:51.000112 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:51.000196 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:51.000270 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:51.000347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:51.000438 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:51.000519 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:51.000593 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] standard_threads = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:51.000666 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:51.000743 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:51.000871 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] standard_threads = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:51.000952 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:51.001026 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:51.001103 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:51.001176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:51.001248 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] use_journal = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:51.001327 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:51.001401 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] use_json = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:51.001472 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] use_journal = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:51.001543 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] use_stderr = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:51.001645 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] use_json = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:51.001721 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] use_syslog = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:51.001802 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] use_stderr = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:51.001882 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:51.001955 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] use_syslog = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:51.002027 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.002098 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 15:49:51.002182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.002263 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.002336 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.002408 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.002478 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.002549 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.002693 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.002777 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.002933 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.003019 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.003182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.003262 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.003423 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.003505 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.003662 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.003741 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.003911 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.003994 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.004160 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.004240 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.004394 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.004479 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.004637 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.004716 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.004873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.004952 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.005111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.005196 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.005357 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.005440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.005623 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.006192 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.006377 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.006464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.006636 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.006754 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.006893 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.006981 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.007144 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.007224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.007384 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.007641 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.007729 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.007908 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.008001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.008161 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.008246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.008412 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.008504 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.008644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.008724 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.008876 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.008955 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.009106 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.009197 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.009341 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.009419 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.009626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.009709 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.009858 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.009940 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.010089 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.010173 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.010318 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.010396 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.010564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.010643 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.010808 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.010887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.011045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.011133 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.011291 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.011371 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.011523 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.011602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.011757 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.011837 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.011987 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.012063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.012211 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.012296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.012440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.012517 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.012669 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.012748 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.012898 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.012976 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.013133 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.013218 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.013380 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.013467 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.013628 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.013709 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.013866 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.013955 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.014096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.014182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.014328 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.014407 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.014563 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.014666 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.014807 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.014917 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.015020 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.015122 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.015260 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.015395 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.015508 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.015614 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.015739 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.015818 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.015936 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.016014 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.016173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.016255 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.016401 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.016483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.016648 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.016775 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.016882 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.016971 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.017117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.017205 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.017345 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.017436 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.017625 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.017748 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.017864 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.018013 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.018094 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.018211 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.018319 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.018399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.018555 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.018636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.018819 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.018899 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.019056 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.019144 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.019303 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.019392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.019545 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.019630 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.019781 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.019860 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.020018 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.020102 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.020253 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.020332 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.020500 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.020590 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.020737 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.020816 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.020971 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.021051 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.021212 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.021300 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.022082 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.022182 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.022281 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.022362 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.022492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.022589 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.022728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.022815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.022959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.023045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.023221 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.023309 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.023461 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.023540 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.023701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.023780 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.023936 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.024093 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.024171 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.024245 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.024398 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.024480 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.024633 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.024725 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.024864 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.024957 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.025094 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.025174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.025322 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.025407 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.025551 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.025703 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.025809 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.025890 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.026037 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.026116 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.026276 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.026365 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.026511 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.026591 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.026758 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.026837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.026991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.027087 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.027247 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.027326 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.027503 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.027582 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.027742 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.027831 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.027972 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.external = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.028051 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.external = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.028201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.028281 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.028493 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.028609 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.028724 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.028805 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.028956 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.password = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.029038 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.password = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.029185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.token = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.029265 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.token = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.029422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.029509 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.029680 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.029763 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.029927 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.030007 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.030171 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.030257 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.030431 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.030524 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.030678 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.030763 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.030909 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.030992 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.031144 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.031224 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.031392 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.031472 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.031637 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.031719 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.031884 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.031972 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.032118 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.032198 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.032361 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.032442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.032594 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.032674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.032850 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.032929 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.033099 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.033181 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.033336 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.033417 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.033635 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.033721 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.033890 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.033975 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.034134 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.034223 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.034394 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.034476 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.034635 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.034719 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.034892 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.034974 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.035130 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.035201 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.035364 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.035446 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.035612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.035691 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.035855 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.035936 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.036086 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.036169 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.036330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.036412 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.036566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.036645 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.036815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.036897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.037086 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.037167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.037340 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.037422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.037589 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.037672 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.037845 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.037925 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.038081 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.038163 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.038320 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.038400 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.038567 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.038646 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.038802 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.038891 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.039039 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.039120 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.039416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.039497 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.039653 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.039742 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.039892 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.039972 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.040145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.040226 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.040398 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.040491 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.040630 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.040715 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.040858 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.040945 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.041088 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.041177 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.041334 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.041416 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.041590 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.041672 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.041851 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.041945 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.042108 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.042190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.042354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.042434 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.042625 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.042716 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.042864 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.042945 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.043102 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.043184 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.043334 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.043424 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.043572 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.043653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.043802 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.043882 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.044042 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.044131 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.044282 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.044362 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.044521 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.044603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.044771 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.044860 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.045001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.045081 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.045247 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.045329 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.045496 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.045603 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.045757 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.045849 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.046005 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.046088 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.046254 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.046347 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.046485 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.046565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.046723 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.046804 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.046954 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.047044 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.047190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.047272 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.047427 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.047518 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.047672 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.047763 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.047919 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.048001 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.048157 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.048237 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.048405 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.048494 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.048653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.048735 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.048885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.048965 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.049114 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.049204 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.049362 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.049477 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.049632 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.049753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.049873 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.049994 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.050110 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.050229 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.050348 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.050465 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.050581 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.050696 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.050823 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.050935 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.051093 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.051175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.051332 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.051416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.051583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.051665 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.051833 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.051914 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.052080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.052163 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.052318 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.052398 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.052565 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.052645 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.052802 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.052885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.053036 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.053117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.053270 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.053352 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.053499 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.053597 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.053769 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.053851 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.054012 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.054094 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.054257 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.054341 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.054502 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.054582 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.054744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.054824 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.054972 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.055055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.055214 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.055295 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.055452 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.055532 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.055683 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.055764 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.055916 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.055997 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.056148 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.056229 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.056391 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.056474 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.056638 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.056718 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.056871 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.056957 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.057121 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.057204 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.057350 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.057430 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.057606 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.057689 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.057845 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.057936 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.058079 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.058161 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.058314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.058395 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.058558 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.058642 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.058800 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.058880 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.059029 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.059109 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.059262 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.059343 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.059503 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.059584 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.059748 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.059828 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.059995 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.060078 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.060232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.060321 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.060477 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.060557 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.060724 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.060805 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.060958 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.061038 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.061203 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.061283 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.061436 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.061513 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.061686 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.061769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.061923 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] role.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.062008 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] role.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.062167 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] role.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.062250 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] role.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.062396 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.062476 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.062642 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.062728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.062889 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.062973 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.063136 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.063218 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.063382 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.063464 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.063628 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.063711 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.063873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.063954 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.064117 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.064210 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.064364 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.064445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.064594 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.064674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.064839 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.064929 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.065086 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.065167 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.065332 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.065412 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.065648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.065747 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.066029 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.066111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.066264 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.066345 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.066513 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.066603 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.066761 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.066840 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.067008 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.067089 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.067256 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.067345 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.067488 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.067568 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.067741 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.067820 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.067989 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.068080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.068749 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.068834 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.068982 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.069064 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.069215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.069305 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.069445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.069525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.069718 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.069801 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.069974 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.070058 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.070221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.070302 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.070458 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.070539 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.070694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.070778 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.070924 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.071005 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.071158 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] token.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.071239 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.071393 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.071476 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.071639 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.071720 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.071874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.071955 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.072129 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.072212 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.072373 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.072453 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.072610 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.072690 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.072849 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.072931 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.073090 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.073170 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.073334 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.073414 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.073569 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.073733 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.073829 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.073909 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.074065 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.074147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.074312 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.074395 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.074557 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.074637 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.074783 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.074863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.075028 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.075110 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.075323 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.075403 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.075582 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.connection = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.075671 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.connection = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.075839 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.075930 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.076071 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.076160 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.076308 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.076390 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.076548 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.076628 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.076781 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.076869 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.077015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.077095 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.077246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.077325 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.077478 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.077595 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.077744 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.077825 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.077978 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.078058 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.078221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.078315 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.078457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.078538 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.078702 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.078783 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.078937 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.079029 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.079182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.079264 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.079413 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.079493 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.079650 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.079739 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.079884 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.079964 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 15:49:51.080039 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 21 15:49:51.080112 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 21 15:49:51.111859 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.112091 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.112323 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.112415 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.112625 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.112852 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.121268 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.121678 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.150246 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.150484 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.150746 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.150981 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.294173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.294493 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.342894 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.343136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.343634 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.343871 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.409066 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.409303 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.409526 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.409662 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.409779 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.409868 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.409999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.410084 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.410223 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.410311 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.410527 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.410749 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.435996 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.436203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.436298 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.436486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.436574 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.436709 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.436792 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.436927 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.437012 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.437142 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.437223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.437356 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.437448 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.437604 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.448515 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.448985 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.461871 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.462107 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.462242 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.462374 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.462500 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.462629 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.462772 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.462895 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.463025 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.463146 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.463263 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.463379 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.463512 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.463512 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 15:49:51.481802 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.481995 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 15:49:51.482125 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.482224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 15:49:51.482336 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.482538 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 15:49:51.482619 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.482755 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 15:49:51.482838 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.483031 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.483115 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.483299 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.483380 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.483554 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.483641 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.483815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.494801 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.495029 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.495112 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.495298 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.495392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.495578 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.495661 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.495881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.503787 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.504083 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.507627 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.507908 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.507995 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.508215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.508313 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.508585 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.508667 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.508872 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.508954 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.509139 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.509231 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.509412 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.509529 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.509710 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.509986 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.510079 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.510263 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.510705 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.510782 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.511017 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.511453 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.511750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.537933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.539654 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.539940 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.544544 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.545414 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.545749 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.546035 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.546272 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.546564 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.552065 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.552368 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.552693 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.553555 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.553882 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.558659 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.558957 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.559243 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.559528 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.559807 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.560082 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.560174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.560350 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.560475 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.560618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.565312 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.565634 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.565928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.566217 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.566809 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.566809 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.567020 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.567292 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.580040 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.580335 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.585485 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.587234 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.587527 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.589130 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.589426 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.592198 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.594473 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.595852 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.596155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.598113 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.601477 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.602187 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.602486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.602775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.603073 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.603351 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.603627 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.605134 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.608946 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.609251 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.609546 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.609864 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.610154 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.610446 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.622586 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.622965 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.623246 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.623513 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.629451 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.629854 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.630132 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.630408 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.631958 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.633687 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.633965 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.637021 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.637304 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.637597 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.637887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.638216 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.638879 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.640592 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.640873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.643965 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.644250 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.644532 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.644817 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.645152 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.651672 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.651960 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.652238 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.652527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.652807 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.653096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.653381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.658631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.658918 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.659202 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.659488 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.659771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.660066 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.660345 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.673487 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.676810 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.680728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.681005 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.682392 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.684073 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.684327 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.684432 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.684629 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.687694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.689306 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.690981 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.691265 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.691556 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.692169 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.693818 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.697052 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.697345 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.697659 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.697949 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.699087 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.700715 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.703967 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.704264 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.704555 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.704844 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.707206 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.707498 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.707779 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.714191 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.714490 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.714771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.715676 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.715958 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.716236 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.716508 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.716776 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.717053 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.717325 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.717617 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.717899 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.718166 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.718441 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.718702 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.718963 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.719227 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.722710 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 15:49:51.722999 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.723284 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.723560 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.723834 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.724108 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.724921 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.725210 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.725485 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.725785 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.726077 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.726343 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.726621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.726888 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 15:49:51.758306 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55cacbf019c0 pid: 80237 (default app) Jul 21 15:49:51.761685 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecdf800-9be6-43ab-9aec-a64eb41c96c3 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:49:51.761984 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecdf800-9be6-43ab-9aec-a64eb41c96c3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:49:51.762270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecdf800-9be6-43ab-9aec-a64eb41c96c3 None None] PATH_INFO: `/v3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:49:51.765754 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55cacbf019c0 pid: 80238 (default app) Jul 21 15:49:51.766439 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1/1] 10.222.0.52 () {50 vars in 791 bytes} [Thu Jul 21 15:49:51 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:49:59.471442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d13e3-5bd9-4aea-83ae-9391b54cd067 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:49:59.472001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d13e3-5bd9-4aea-83ae-9391b54cd067 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:49:59.472001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d13e3-5bd9-4aea-83ae-9391b54cd067 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:49:59.476605 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1/2] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:49:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:49:59.481896 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fb23e-5d3c-439a-a44a-61d6afad2c2a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:49:59.482274 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fb23e-5d3c-439a-a44a-61d6afad2c2a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:49:59.482458 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fb23e-5d3c-439a-a44a-61d6afad2c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:49:59.551559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: INFO dbcounter [None req-068fb23e-5d3c-439a-a44a-61d6afad2c2a None None] Registered counter for database keystone Jul 21 15:49:59.573711 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG oslo_db.sqlalchemy.engines [None req-068fb23e-5d3c-439a-a44a-61d6afad2c2a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80237) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 21 15:49:59.580119 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writer thread running {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 21 15:49:59.753724 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG passlib.handlers.bcrypt [None req-068fb23e-5d3c-439a-a44a-61d6afad2c2a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80237) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 21 15:49:59.754140 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG passlib.handlers.bcrypt [None req-068fb23e-5d3c-439a-a44a-61d6afad2c2a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80237) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 21 15:49:59.809846 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-068fb23e-5d3c-439a-a44a-61d6afad2c2a None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:49:59.811419 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-068fb23e-5d3c-439a-a44a-61d6afad2c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:49:59.933637 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 2/3] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:49:59 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 453 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 21 15:50:00.075477 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d13e3-5bd9-4aea-83ae-9391b54cd067 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:00.144822 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: INFO dbcounter [None req-459d13e3-5bd9-4aea-83ae-9391b54cd067 None None] Registered counter for database keystone Jul 21 15:50:00.165015 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG oslo_db.sqlalchemy.engines [None req-459d13e3-5bd9-4aea-83ae-9391b54cd067 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80238) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 21 15:50:00.171135 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writer thread running {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 21 15:50:00.290365 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d13e3-5bd9-4aea-83ae-9391b54cd067 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:00.294604 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2e641-1c81-4f8d-961a-9431fd293eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:00.295264 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2e641-1c81-4f8d-961a-9431fd293eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:00.295548 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2e641-1c81-4f8d-961a-9431fd293eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:00.295823 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2e641-1c81-4f8d-961a-9431fd293eb1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:00.304058 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2e641-1c81-4f8d-961a-9431fd293eb1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:00.357556 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3ec2e641-1c81-4f8d-961a-9431fd293eb1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 21 15:50:00.357834 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2e641-1c81-4f8d-961a-9431fd293eb1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:00.368862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 2/4] 10.222.0.52 () {58 vars in 1324 bytes} [Thu Jul 21 15:50:00 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 15:50:00.374858 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fb23e-5d3c-439a-a44a-61d6afad2c2a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:00.389448 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fb23e-5d3c-439a-a44a-61d6afad2c2a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:00.393883 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9009dec-1af9-42c2-80a7-2c253405bccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:00.395023 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9009dec-1af9-42c2-80a7-2c253405bccd admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:00.395313 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9009dec-1af9-42c2-80a7-2c253405bccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:00.395591 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9009dec-1af9-42c2-80a7-2c253405bccd admin admin] PATH_INFO: `/v3/projects/f2cb644b2d994629b4b82065db94ebac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:00.400251 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9009dec-1af9-42c2-80a7-2c253405bccd admin admin] RBAC: Authorizing `identity:get_project(project_id=f2cb644b2d994629b4b82065db94ebac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:00.450954 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e9009dec-1af9-42c2-80a7-2c253405bccd admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 21 15:50:00.451306 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9009dec-1af9-42c2-80a7-2c253405bccd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:00.455963 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 3/5] 10.222.0.52 () {56 vars in 1207 bytes} [Thu Jul 21 15:50:00 2022] GET /identity/v3/projects/f2cb644b2d994629b4b82065db94ebac => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:50:02.459967 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741c12d-f2fc-4a15-8472-db28d91d135f None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:02.460469 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741c12d-f2fc-4a15-8472-db28d91d135f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:02.460557 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741c12d-f2fc-4a15-8472-db28d91d135f None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:02.464671 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 3/6] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 15:50:02.469954 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bbfd2d-536d-4a03-8a04-303fd4932c48 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:02.470374 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bbfd2d-536d-4a03-8a04-303fd4932c48 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:02.470524 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bbfd2d-536d-4a03-8a04-303fd4932c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:02.489962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e7bbfd2d-536d-4a03-8a04-303fd4932c48 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:02.491433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e7bbfd2d-536d-4a03-8a04-303fd4932c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:02.499276 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 4/7] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:02 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 21 15:50:02.521771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741c12d-f2fc-4a15-8472-db28d91d135f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:02.533733 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741c12d-f2fc-4a15-8472-db28d91d135f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:02.538515 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f56e738-8d69-4791-a7b5-4cfd1482eb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:02.539147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56e738-8d69-4791-a7b5-4cfd1482eb18 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:02.539433 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56e738-8d69-4791-a7b5-4cfd1482eb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:02.539713 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56e738-8d69-4791-a7b5-4cfd1482eb18 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:02.547424 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f56e738-8d69-4791-a7b5-4cfd1482eb18 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:02.549206 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f56e738-8d69-4791-a7b5-4cfd1482eb18 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:02.559104 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 4/8] 10.222.0.52 () {58 vars in 1324 bytes} [Thu Jul 21 15:50:02 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 15:50:02.564889 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bbfd2d-536d-4a03-8a04-303fd4932c48 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:02.574085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bbfd2d-536d-4a03-8a04-303fd4932c48 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:02.579110 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9bedd6-0089-40b1-b0ec-bb6d812fe708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:02.579973 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bedd6-0089-40b1-b0ec-bb6d812fe708 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:02.580263 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bedd6-0089-40b1-b0ec-bb6d812fe708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:02.580546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bedd6-0089-40b1-b0ec-bb6d812fe708 admin admin] PATH_INFO: `/v3/users/bf570352584f4974b4b425f3e760bf0b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:02.582031 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9bedd6-0089-40b1-b0ec-bb6d812fe708 admin admin] RBAC: Authorizing `identity:get_user(user_id=bf570352584f4974b4b425f3e760bf0b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:02.584392 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9bedd6-0089-40b1-b0ec-bb6d812fe708 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:02.586015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 5/9] 10.222.0.52 () {56 vars in 1198 bytes} [Thu Jul 21 15:50:02 2022] GET /identity/v3/users/bf570352584f4974b4b425f3e760bf0b => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:04.645204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ca6ea-0f83-4e09-8940-a9ddfb5e950b None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:04.646264 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ca6ea-0f83-4e09-8940-a9ddfb5e950b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:04.646264 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ca6ea-0f83-4e09-8940-a9ddfb5e950b None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:04.647569 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 5/10] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 15:50:04.652938 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9d0fb-adb7-44ea-82d0-6423f0a14f6c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:04.653321 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9d0fb-adb7-44ea-82d0-6423f0a14f6c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:04.653504 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9d0fb-adb7-44ea-82d0-6423f0a14f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:04.670776 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d8b9d0fb-adb7-44ea-82d0-6423f0a14f6c None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:04.671971 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d8b9d0fb-adb7-44ea-82d0-6423f0a14f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:04.694497 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51848c1-be09-4f76-873f-7779fb82f3b5 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:04.694789 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51848c1-be09-4f76-873f-7779fb82f3b5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:04.695196 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51848c1-be09-4f76-873f-7779fb82f3b5 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:04.696541 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 6/11] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:04.701556 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d17df-a37c-46f2-a40b-ec9b9b4abb2c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:04.701873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d17df-a37c-46f2-a40b-ec9b9b4abb2c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:04.702149 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d17df-a37c-46f2-a40b-ec9b9b4abb2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:04.743956 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG passlib.handlers.bcrypt [None req-e75d17df-a37c-46f2-a40b-ec9b9b4abb2c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80238) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 21 15:50:04.744383 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG passlib.handlers.bcrypt [None req-e75d17df-a37c-46f2-a40b-ec9b9b4abb2c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80238) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 21 15:50:04.754360 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 6/12] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:04.777483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9d0fb-adb7-44ea-82d0-6423f0a14f6c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:04.790629 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e75d17df-a37c-46f2-a40b-ec9b9b4abb2c None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:04.792249 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e75d17df-a37c-46f2-a40b-ec9b9b4abb2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:04.801391 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:04.884822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9d0fb-adb7-44ea-82d0-6423f0a14f6c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:04.892407 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 7/13] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 15:50:04.916500 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d17df-a37c-46f2-a40b-ec9b9b4abb2c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:04.919162 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d23c73-df97-44fa-a183-eeed213a6eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:04.919971 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d23c73-df97-44fa-a183-eeed213a6eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:04.920309 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d23c73-df97-44fa-a183-eeed213a6eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:04.920593 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d23c73-df97-44fa-a183-eeed213a6eb7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:04.932554 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d23c73-df97-44fa-a183-eeed213a6eb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:04.943752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-39d23c73-df97-44fa-a183-eeed213a6eb7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:50:04.943752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:04.943752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:04.943752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:04.943752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:04.943752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:04.943752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:04.943752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:04.943752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:04.943752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:04.943752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:04.943752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:04.943752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:04.943752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:04.943752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:04.943752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:04.944700 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:04.944700 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:04.944700 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:04.944700 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:04.944700 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:04.944700 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:04.944700 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:04.944700 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:04.944700 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:04.944700 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:04.944700 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:04.944700 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:04.944700 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:04.944700 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:04.944700 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:04.944700 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:04.945624 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:04.945624 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:04.945624 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:04.945624 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:04.945624 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:04.945624 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:04.945624 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:04.945624 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:04.945624 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:04.945624 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:04.945624 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:50:04.945624 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:50:04.945624 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 7/14] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:50:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:50:04.951260 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d23c73-df97-44fa-a183-eeed213a6eb7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:05.028970 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d17df-a37c-46f2-a40b-ec9b9b4abb2c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:05.031311 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:05.034729 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:05.055406 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d23c73-df97-44fa-a183-eeed213a6eb7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:05.068378 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea70820d-d9d6-4a60-ae6c-f7078c7dad17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:05.069280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70820d-d9d6-4a60-ae6c-f7078c7dad17 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:05.069567 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70820d-d9d6-4a60-ae6c-f7078c7dad17 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:05.069873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70820d-d9d6-4a60-ae6c-f7078c7dad17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:05.089931 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12cf47-e2cb-4d17-aad5-358b157aadac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:05.090601 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12cf47-e2cb-4d17-aad5-358b157aadac None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:05.090888 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12cf47-e2cb-4d17-aad5-358b157aadac None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:05.091170 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12cf47-e2cb-4d17-aad5-358b157aadac None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:05.091929 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f12cf47-e2cb-4d17-aad5-358b157aadac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:05.092239 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f12cf47-e2cb-4d17-aad5-358b157aadac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:05.093697 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f12cf47-e2cb-4d17-aad5-358b157aadac None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:05.105947 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 8/15] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:50:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:50:05.112122 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12cf47-e2cb-4d17-aad5-358b157aadac None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:05.138801 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea70820d-d9d6-4a60-ae6c-f7078c7dad17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:05.140344 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea70820d-d9d6-4a60-ae6c-f7078c7dad17 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:05.212643 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12cf47-e2cb-4d17-aad5-358b157aadac None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:05.244823 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 8/16] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:50:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 15:50:05.247083 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b25fa-d3d7-483f-8315-d56a87398080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:05.247967 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b25fa-d3d7-483f-8315-d56a87398080 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:05.248253 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b25fa-d3d7-483f-8315-d56a87398080 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:05.248543 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b25fa-d3d7-483f-8315-d56a87398080 None admin] PATH_INFO: `/v3/users/bf570352584f4974b4b425f3e760bf0b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:05.249785 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677b25fa-d3d7-483f-8315-d56a87398080 None admin] RBAC: Authorizing `identity:get_user(user_id=bf570352584f4974b4b425f3e760bf0b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:05.250772 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea70820d-d9d6-4a60-ae6c-f7078c7dad17 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:05.251300 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677b25fa-d3d7-483f-8315-d56a87398080 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:05.252934 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 9/17] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:05 2022] GET /identity/v3/users/bf570352584f4974b4b425f3e760bf0b => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:50:05.254162 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:05.259062 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b25fa-d3d7-483f-8315-d56a87398080 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:05.274921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:05.362224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea70820d-d9d6-4a60-ae6c-f7078c7dad17 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:05.371890 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b25fa-d3d7-483f-8315-d56a87398080 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:05.396623 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dcc93c-7880-48dd-ae99-9b4136758dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:05.397415 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dcc93c-7880-48dd-ae99-9b4136758dea None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:05.397735 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dcc93c-7880-48dd-ae99-9b4136758dea None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:05.398017 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dcc93c-7880-48dd-ae99-9b4136758dea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:05.406167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614dd02f-2ed7-453e-8206-ef236cade09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:05.406954 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614dd02f-2ed7-453e-8206-ef236cade09e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:05.407238 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614dd02f-2ed7-453e-8206-ef236cade09e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:05.407513 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614dd02f-2ed7-453e-8206-ef236cade09e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:05.408695 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614dd02f-2ed7-453e-8206-ef236cade09e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:05.410186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614dd02f-2ed7-453e-8206-ef236cade09e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:05.411959 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 10/18] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:50:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:50:05.417789 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614dd02f-2ed7-453e-8206-ef236cade09e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:05.417993 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dcc93c-7880-48dd-ae99-9b4136758dea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:05.419456 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dcc93c-7880-48dd-ae99-9b4136758dea None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:05.444837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-a8dcc93c-7880-48dd-ae99-9b4136758dea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:50:05.444837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:05.444837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:05.444837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:05.444837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:05.444837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:05.444837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:05.444837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:05.444837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:05.444837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:05.444837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:05.444837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:05.444837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:50:05.444837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:50:05.444837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:50:05.445739 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:50:05.445739 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:05.445739 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:05.445739 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:05.445739 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:05.445739 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:05.445739 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:05.445739 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:05.445739 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:05.445739 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:05.445739 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:05.445739 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:05.445739 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:05.445739 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:05.445739 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:05.445739 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:05.445739 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:05.446565 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:50:05.446565 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:50:05.446565 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:50:05.446565 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:50:05.446565 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 9/19] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:50:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:50:05.451742 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dcc93c-7880-48dd-ae99-9b4136758dea None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:05.500502 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:05.507770 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:05.521028 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614dd02f-2ed7-453e-8206-ef236cade09e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:05.551336 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dcc93c-7880-48dd-ae99-9b4136758dea None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:05.554100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b098874-f0e1-4d29-a776-9793ed175b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:05.554764 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b098874-f0e1-4d29-a776-9793ed175b8d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:05.555049 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b098874-f0e1-4d29-a776-9793ed175b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:05.555331 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b098874-f0e1-4d29-a776-9793ed175b8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:05.556169 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b098874-f0e1-4d29-a776-9793ed175b8d None admin] RBAC: Adding query filter params (role.id=475521b8a0174037b162f92022aea1a1, scope.domain.id=default, user.id=bf570352584f4974b4b425f3e760bf0b) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:05.556485 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b098874-f0e1-4d29-a776-9793ed175b8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:05.557966 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b098874-f0e1-4d29-a776-9793ed175b8d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:05.564871 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 11/20] 10.222.0.52 () {56 vars in 1322 bytes} [Thu Jul 21 15:50:05 2022] GET /identity/v3/role_assignments?user.id=bf570352584f4974b4b425f3e760bf0b&scope.domain.id=default&role.id=475521b8a0174037b162f92022aea1a1 => generated 233 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:50:05.587953 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce520a-c26c-4297-8fe8-ad4ebfa9dd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:05.588643 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce520a-c26c-4297-8fe8-ad4ebfa9dd26 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:05.588932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce520a-c26c-4297-8fe8-ad4ebfa9dd26 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:05.589214 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce520a-c26c-4297-8fe8-ad4ebfa9dd26 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:05.590016 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce520a-c26c-4297-8fe8-ad4ebfa9dd26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:05.590330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce520a-c26c-4297-8fe8-ad4ebfa9dd26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:05.591798 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce520a-c26c-4297-8fe8-ad4ebfa9dd26 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:05.611517 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 10/21] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:50:08.028082 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6369235-a1a2-49e9-84b7-77dbbbf3f63c None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:08.028980 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6369235-a1a2-49e9-84b7-77dbbbf3f63c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:08.029544 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6369235-a1a2-49e9-84b7-77dbbbf3f63c None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:08.031114 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 12/22] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:08.037048 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2109e6-517b-4dd8-8d94-b57eafbe6f0b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:08.037451 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2109e6-517b-4dd8-8d94-b57eafbe6f0b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:08.037746 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2109e6-517b-4dd8-8d94-b57eafbe6f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:08.066718 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1f2109e6-517b-4dd8-8d94-b57eafbe6f0b None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:08.068490 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1f2109e6-517b-4dd8-8d94-b57eafbe6f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:08.158543 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 11/23] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:08.201837 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6369235-a1a2-49e9-84b7-77dbbbf3f63c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:08.350895 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6369235-a1a2-49e9-84b7-77dbbbf3f63c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:08.390430 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439fc41-f3e9-473d-9594-22230318fc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:08.391246 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439fc41-f3e9-473d-9594-22230318fc7c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:08.391537 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439fc41-f3e9-473d-9594-22230318fc7c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:08.391816 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439fc41-f3e9-473d-9594-22230318fc7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:08.408473 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5439fc41-f3e9-473d-9594-22230318fc7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:08.416565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-5439fc41-f3e9-473d-9594-22230318fc7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:50:08.416565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:08.416565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:08.416565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:08.416565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:08.416565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:08.416565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:08.416565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:08.416565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:08.416565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:08.416565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:08.416565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:08.416565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:08.416565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:08.416565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:08.416565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:08.417453 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:08.417453 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:08.417453 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:08.417453 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:08.417453 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:08.417453 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:08.417453 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:08.417453 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:08.417453 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:08.417453 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:08.417453 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:08.417453 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:08.417453 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:08.417453 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:08.417453 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:08.417453 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:08.418296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:08.418296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:08.418296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:08.418296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:08.418296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:08.418296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:08.418296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:08.418296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:08.418296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:08.418296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:08.418296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:50:08.418296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:50:08.418296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 13/24] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:50:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:08.427216 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2109e6-517b-4dd8-8d94-b57eafbe6f0b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:08.455518 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50824a1-657a-4fd8-bf31-5ac9f4f73405 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:08.455829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50824a1-657a-4fd8-bf31-5ac9f4f73405 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:08.456097 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50824a1-657a-4fd8-bf31-5ac9f4f73405 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:08.458555 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 14/25] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:08.463724 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5cfe3-5106-4acf-8b6b-3f2c480bd879 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:08.464013 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5cfe3-5106-4acf-8b6b-3f2c480bd879 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:08.464294 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5cfe3-5106-4acf-8b6b-3f2c480bd879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:08.474755 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:08.491462 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d9a5cfe3-5106-4acf-8b6b-3f2c480bd879 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:08.492722 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d9a5cfe3-5106-4acf-8b6b-3f2c480bd879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:08.525425 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:08.628752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 15/26] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 15:50:08.630187 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694a566-9bd7-4c16-930d-5808ab52991c None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:08.630434 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694a566-9bd7-4c16-930d-5808ab52991c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:08.630723 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694a566-9bd7-4c16-930d-5808ab52991c None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:08.632017 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 16/27] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:08.638927 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba0321-12bd-4842-ab3c-6553ae08f6a6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:08.639222 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba0321-12bd-4842-ab3c-6553ae08f6a6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:08.639508 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba0321-12bd-4842-ab3c-6553ae08f6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:08.639665 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2109e6-517b-4dd8-8d94-b57eafbe6f0b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:08.669671 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-74ba0321-12bd-4842-ab3c-6553ae08f6a6 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:08.670893 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-74ba0321-12bd-4842-ab3c-6553ae08f6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:08.685154 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd5545-e5c6-46a4-b6fa-5d43296d3e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:08.685881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd5545-e5c6-46a4-b6fa-5d43296d3e87 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:08.686174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd5545-e5c6-46a4-b6fa-5d43296d3e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:08.686462 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd5545-e5c6-46a4-b6fa-5d43296d3e87 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:08.687239 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd5545-e5c6-46a4-b6fa-5d43296d3e87 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:08.687554 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd5545-e5c6-46a4-b6fa-5d43296d3e87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:08.688967 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd5545-e5c6-46a4-b6fa-5d43296d3e87 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:08.710729 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 12/28] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:50:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:08.711570 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd5545-e5c6-46a4-b6fa-5d43296d3e87 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:08.778248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 17/29] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:08.779003 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba0321-12bd-4842-ab3c-6553ae08f6a6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:08.831627 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd5545-e5c6-46a4-b6fa-5d43296d3e87 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:08.841677 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:08.869558 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e957a1-e2cf-416a-9ff8-907acbbdd6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:08.870259 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e957a1-e2cf-416a-9ff8-907acbbdd6de None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:08.870552 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e957a1-e2cf-416a-9ff8-907acbbdd6de None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:08.870840 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e957a1-e2cf-416a-9ff8-907acbbdd6de None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:08.871604 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e957a1-e2cf-416a-9ff8-907acbbdd6de None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:08.873023 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e957a1-e2cf-416a-9ff8-907acbbdd6de None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:08.881534 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:08.909487 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba0321-12bd-4842-ab3c-6553ae08f6a6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:08.952735 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3933e2d4-4157-4e49-8f73-d1d6467fb2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:08.953658 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933e2d4-4157-4e49-8f73-d1d6467fb2dd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:08.953948 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933e2d4-4157-4e49-8f73-d1d6467fb2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:08.954231 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933e2d4-4157-4e49-8f73-d1d6467fb2dd None admin] PATH_INFO: `/v3/users/bf570352584f4974b4b425f3e760bf0b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:08.955447 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3933e2d4-4157-4e49-8f73-d1d6467fb2dd None admin] RBAC: Authorizing `identity:get_user(user_id=bf570352584f4974b4b425f3e760bf0b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:08.957220 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3933e2d4-4157-4e49-8f73-d1d6467fb2dd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:08.959187 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 18/30] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:08 2022] GET /identity/v3/users/bf570352584f4974b4b425f3e760bf0b => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:08.959823 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3933e2d4-4157-4e49-8f73-d1d6467fb2dd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:09.018665 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 13/31] 10.222.0.52 () {60 vars in 1131 bytes} [Thu Jul 21 15:50:08 2022] POST /identity/v3/roles => generated 219 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:50:09.020305 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2aa24-18d5-473b-ab61-e1064a4d0070 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:09.020602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2aa24-18d5-473b-ab61-e1064a4d0070 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:09.020892 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2aa24-18d5-473b-ab61-e1064a4d0070 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:09.022554 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 14/32] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:09.023300 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d2aa24-18d5-473b-ab61-e1064a4d0070 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:09.103880 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3933e2d4-4157-4e49-8f73-d1d6467fb2dd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:09.151952 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d2aa24-18d5-473b-ab61-e1064a4d0070 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:09.153978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:09.161902 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a07fd-cfc7-4720-a645-65fdfd270eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:09.162907 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a07fd-cfc7-4720-a645-65fdfd270eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:09.163201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a07fd-cfc7-4720-a645-65fdfd270eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:09.163484 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a07fd-cfc7-4720-a645-65fdfd270eb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:09.176249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9a07fd-cfc7-4720-a645-65fdfd270eb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:09.177891 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9a07fd-cfc7-4720-a645-65fdfd270eb6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:09.198264 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-5b9a07fd-cfc7-4720-a645-65fdfd270eb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:50:09.198264 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:09.198264 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:09.198264 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:09.198264 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:09.198264 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:09.198264 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:09.198264 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:09.198264 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:09.198264 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:09.198264 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:09.198264 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:09.198264 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:50:09.198264 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:50:09.198264 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:50:09.200273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:50:09.200273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:09.200273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:09.200273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:09.200273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:09.200273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:09.200273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:09.200273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:09.200273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:09.200273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:09.200273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:09.200273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:09.200273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:09.200273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:09.200273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:09.200273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:09.200273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:09.202317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:50:09.202317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:50:09.202317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:50:09.202317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:50:09.202797 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b0e3b9-5fb1-498c-82d6-363dd26c31fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:09.202957 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 19/33] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:50:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:09.203106 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0e3b9-5fb1-498c-82d6-363dd26c31fa None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:09.203106 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0e3b9-5fb1-498c-82d6-363dd26c31fa None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:09.203106 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0e3b9-5fb1-498c-82d6-363dd26c31fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:09.203495 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e665081-a9c6-49b4-8d7c-05227da9ff6c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:09.203495 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e665081-a9c6-49b4-8d7c-05227da9ff6c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:09.203495 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e665081-a9c6-49b4-8d7c-05227da9ff6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:09.203900 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b0e3b9-5fb1-498c-82d6-363dd26c31fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:09.204037 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b0e3b9-5fb1-498c-82d6-363dd26c31fa None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:09.209479 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 15/34] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:50:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:09.209479 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea8039-8aca-4d4d-9e0e-58b1b2efc11f None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:09.209479 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea8039-8aca-4d4d-9e0e-58b1b2efc11f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:09.209479 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea8039-8aca-4d4d-9e0e-58b1b2efc11f None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:09.212474 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 16/35] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:09.212474 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea8039-8aca-4d4d-9e0e-58b1b2efc11f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:09.229857 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4e665081-a9c6-49b4-8d7c-05227da9ff6c None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:09.234741 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4e665081-a9c6-49b4-8d7c-05227da9ff6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:09.264184 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:09.326699 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 20/36] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:09.328119 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a03b54-80e0-4ba0-bd94-a08a1aba4f4e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:09.328422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a03b54-80e0-4ba0-bd94-a08a1aba4f4e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:09.328708 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a03b54-80e0-4ba0-bd94-a08a1aba4f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:09.351553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-65a03b54-80e0-4ba0-bd94-a08a1aba4f4e None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:09.352725 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-65a03b54-80e0-4ba0-bd94-a08a1aba4f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:09.358264 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea8039-8aca-4d4d-9e0e-58b1b2efc11f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:09.395003 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e06d0-eaf1-413d-9543-4b42c15e9c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:09.396193 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e06d0-eaf1-413d-9543-4b42c15e9c62 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:09.396895 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e06d0-eaf1-413d-9543-4b42c15e9c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:09.397186 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e06d0-eaf1-413d-9543-4b42c15e9c62 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:09.397976 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753e06d0-eaf1-413d-9543-4b42c15e9c62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:09.398315 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753e06d0-eaf1-413d-9543-4b42c15e9c62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:09.400415 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753e06d0-eaf1-413d-9543-4b42c15e9c62 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:09.412804 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 17/37] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:09.418082 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e06d0-eaf1-413d-9543-4b42c15e9c62 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:09.424315 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 21/38] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:09.425294 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a03b54-80e0-4ba0-bd94-a08a1aba4f4e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:09.484386 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:09.523874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:09.531956 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e06d0-eaf1-413d-9543-4b42c15e9c62 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:09.545975 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a03b54-80e0-4ba0-bd94-a08a1aba4f4e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:09.572287 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70beb1-30e3-49ca-9e8d-d1aa8189d25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:09.573177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70beb1-30e3-49ca-9e8d-d1aa8189d25c None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:09.573468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70beb1-30e3-49ca-9e8d-d1aa8189d25c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:09.573788 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70beb1-30e3-49ca-9e8d-d1aa8189d25c None admin] PATH_INFO: `/v3/domains/default/users/bf570352584f4974b4b425f3e760bf0b/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:09.576296 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70beb1-30e3-49ca-9e8d-d1aa8189d25c None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=bf570352584f4974b4b425f3e760bf0b, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:09.577801 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70beb1-30e3-49ca-9e8d-d1aa8189d25c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:09.583554 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2381d17-fec5-4b51-8cca-85126fc2600d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:09.584322 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2381d17-fec5-4b51-8cca-85126fc2600d None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:09.584564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2381d17-fec5-4b51-8cca-85126fc2600d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:09.584869 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2381d17-fec5-4b51-8cca-85126fc2600d None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:09.585615 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2381d17-fec5-4b51-8cca-85126fc2600d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:09.587017 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2381d17-fec5-4b51-8cca-85126fc2600d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:09.596543 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 18/39] 10.222.0.52 () {58 vars in 1361 bytes} [Thu Jul 21 15:50:09 2022] PUT /identity/v3/domains/default/users/bf570352584f4974b4b425f3e760bf0b/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 15:50:09.597391 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70beb1-30e3-49ca-9e8d-d1aa8189d25c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:09.714700 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70beb1-30e3-49ca-9e8d-d1aa8189d25c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:09.761918 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80160391-4c03-4ed2-91d4-2a395de8749b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:09.762690 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80160391-4c03-4ed2-91d4-2a395de8749b None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:09.762984 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80160391-4c03-4ed2-91d4-2a395de8749b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:09.763297 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80160391-4c03-4ed2-91d4-2a395de8749b None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:09.764060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80160391-4c03-4ed2-91d4-2a395de8749b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:09.765615 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80160391-4c03-4ed2-91d4-2a395de8749b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:09.798074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 19/40] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 21 15:50:09 2022] POST /identity/v3/projects => generated 301 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:50:09.798949 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80160391-4c03-4ed2-91d4-2a395de8749b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:09.819654 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:09.829634 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 22/41] 10.222.0.52 () {60 vars in 1131 bytes} [Thu Jul 21 15:50:09 2022] POST /identity/v3/roles => generated 225 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:50:09.831336 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012d9e4-bbfb-4928-a0db-6246e6971527 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:09.831633 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012d9e4-bbfb-4928-a0db-6246e6971527 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:09.831939 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012d9e4-bbfb-4928-a0db-6246e6971527 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:09.833866 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 23/42] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:09.839861 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c4da85-0ea6-49c9-b580-c0577a0bb865 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:09.840154 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c4da85-0ea6-49c9-b580-c0577a0bb865 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:09.840441 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c4da85-0ea6-49c9-b580-c0577a0bb865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:09.860498 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f7c4da85-0ea6-49c9-b580-c0577a0bb865 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:09.862104 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f7c4da85-0ea6-49c9-b580-c0577a0bb865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:09.979785 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 24/43] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 15:50:09.987545 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80160391-4c03-4ed2-91d4-2a395de8749b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:10.025236 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c4da85-0ea6-49c9-b580-c0577a0bb865 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:10.055197 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea3602-a23c-40bc-b78f-02f4bbda69ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:10.056653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea3602-a23c-40bc-b78f-02f4bbda69ab None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:10.056653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea3602-a23c-40bc-b78f-02f4bbda69ab None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:10.056653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea3602-a23c-40bc-b78f-02f4bbda69ab None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:10.057343 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea3602-a23c-40bc-b78f-02f4bbda69ab None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:10.058803 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea3602-a23c-40bc-b78f-02f4bbda69ab None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:10.074631 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:10.077950 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 20/44] 10.222.0.52 () {60 vars in 1131 bytes} [Thu Jul 21 15:50:09 2022] POST /identity/v3/roles => generated 223 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:50:10.178289 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c4da85-0ea6-49c9-b580-c0577a0bb865 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:10.222235 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dcdb4b-8716-4310-ae79-8a3d7fbd8c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:10.223057 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dcdb4b-8716-4310-ae79-8a3d7fbd8c1a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:10.223430 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dcdb4b-8716-4310-ae79-8a3d7fbd8c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:10.223718 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dcdb4b-8716-4310-ae79-8a3d7fbd8c1a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:10.225213 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dcdb4b-8716-4310-ae79-8a3d7fbd8c1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:10.226883 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dcdb4b-8716-4310-ae79-8a3d7fbd8c1a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:10.228781 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 25/45] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:50:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:10.235970 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea3602-a23c-40bc-b78f-02f4bbda69ab None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:10.314606 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:10.347516 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea3602-a23c-40bc-b78f-02f4bbda69ab None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:10.381485 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfeec96-ef28-4ef0-9f95-af8e4a890e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:10.382183 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfeec96-ef28-4ef0-9f95-af8e4a890e46 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:10.382476 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfeec96-ef28-4ef0-9f95-af8e4a890e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:10.382763 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfeec96-ef28-4ef0-9f95-af8e4a890e46 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:10.383499 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfeec96-ef28-4ef0-9f95-af8e4a890e46 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:10.384974 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfeec96-ef28-4ef0-9f95-af8e4a890e46 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:10.410499 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 21/46] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 21 15:50:10 2022] POST /identity/v3/projects => generated 312 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:50:11.668097 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a755a0-3da6-403a-9be5-cbbc52273edf None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:11.668585 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a755a0-3da6-403a-9be5-cbbc52273edf None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:11.668585 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a755a0-3da6-403a-9be5-cbbc52273edf None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:11.669991 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 26/47] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:11.675156 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5af36-6904-4422-b68e-1a6604a8194f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:11.675611 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5af36-6904-4422-b68e-1a6604a8194f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:11.675959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5af36-6904-4422-b68e-1a6604a8194f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:11.694438 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-05b5af36-6904-4422-b68e-1a6604a8194f None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:11.695950 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-05b5af36-6904-4422-b68e-1a6604a8194f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:11.760511 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 22/48] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:11.782926 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a755a0-3da6-403a-9be5-cbbc52273edf None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:11.935846 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a755a0-3da6-403a-9be5-cbbc52273edf None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:11.938424 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:11.971882 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15dc7b9-67de-4f9e-9bec-ddb49b86350e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:11.972678 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15dc7b9-67de-4f9e-9bec-ddb49b86350e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:11.972969 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15dc7b9-67de-4f9e-9bec-ddb49b86350e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:11.973767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15dc7b9-67de-4f9e-9bec-ddb49b86350e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:11.984157 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15dc7b9-67de-4f9e-9bec-ddb49b86350e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:11.992319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-a15dc7b9-67de-4f9e-9bec-ddb49b86350e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:50:11.992319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:11.992319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:11.992319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:11.992319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:11.992319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:11.992319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:11.992319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:11.992319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:11.992319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:11.992319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:11.992319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:11.992319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:11.992319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:11.992319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:11.992319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:11.993233 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:11.993233 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:11.993233 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:11.993233 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:11.993233 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:11.993233 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:11.993233 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:11.993233 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:11.993233 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:11.993233 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:11.993233 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:11.993233 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:11.993233 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:11.993233 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:11.993233 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:11.993233 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:11.994073 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:11.994073 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:11.994073 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:11.994073 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:11.994073 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:11.994073 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:11.994073 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:11.994073 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:11.994073 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:11.994073 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:11.994073 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:50:11.994073 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:50:11.994073 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 27/49] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:50:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:11.999484 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5af36-6904-4422-b68e-1a6604a8194f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:12.047828 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:12.099443 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5af36-6904-4422-b68e-1a6604a8194f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:12.135053 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3826d6-3cc3-485a-a7ca-59a96df63902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:12.136243 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3826d6-3cc3-485a-a7ca-59a96df63902 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:12.136560 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3826d6-3cc3-485a-a7ca-59a96df63902 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:12.136841 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3826d6-3cc3-485a-a7ca-59a96df63902 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:12.137650 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3826d6-3cc3-485a-a7ca-59a96df63902 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:12.137969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3826d6-3cc3-485a-a7ca-59a96df63902 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:12.139644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3826d6-3cc3-485a-a7ca-59a96df63902 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:12.150769 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 23/50] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:50:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:12.156170 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15dc7b9-67de-4f9e-9bec-ddb49b86350e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:12.257940 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15dc7b9-67de-4f9e-9bec-ddb49b86350e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:12.292195 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6df85-20d1-498a-bf6f-1cae9874f0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:12.293070 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6df85-20d1-498a-bf6f-1cae9874f0e8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:12.293359 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6df85-20d1-498a-bf6f-1cae9874f0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:12.293674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6df85-20d1-498a-bf6f-1cae9874f0e8 None admin] PATH_INFO: `/v3/users/bf570352584f4974b4b425f3e760bf0b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:12.294880 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6df85-20d1-498a-bf6f-1cae9874f0e8 None admin] RBAC: Authorizing `identity:get_user(user_id=bf570352584f4974b4b425f3e760bf0b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:12.296351 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6df85-20d1-498a-bf6f-1cae9874f0e8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:12.298036 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 28/51] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:12 2022] GET /identity/v3/users/bf570352584f4974b4b425f3e760bf0b => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:12.303499 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3826d6-3cc3-485a-a7ca-59a96df63902 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:12.482613 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500266a9-e56b-48e7-b4cc-12d5ce473ad5 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:12.482899 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500266a9-e56b-48e7-b4cc-12d5ce473ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:12.483179 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500266a9-e56b-48e7-b4cc-12d5ce473ad5 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:12.484561 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 29/52] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 15:50:12.490353 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef5886-5ef9-4024-b599-0180bac649ee None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:12.490590 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef5886-5ef9-4024-b599-0180bac649ee None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:12.490871 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef5886-5ef9-4024-b599-0180bac649ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:12.507578 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d0ef5886-5ef9-4024-b599-0180bac649ee None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:12.508783 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d0ef5886-5ef9-4024-b599-0180bac649ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:12.509932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3826d6-3cc3-485a-a7ca-59a96df63902 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:12.523574 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:12.533867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:12.546531 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3006a-9b41-4295-9cbe-f7c9a0f65aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:12.547333 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3006a-9b41-4295-9cbe-f7c9a0f65aed None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:12.547623 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3006a-9b41-4295-9cbe-f7c9a0f65aed None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:12.547904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3006a-9b41-4295-9cbe-f7c9a0f65aed None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:12.549097 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3006a-9b41-4295-9cbe-f7c9a0f65aed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:12.550603 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3006a-9b41-4295-9cbe-f7c9a0f65aed None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:12.552806 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 24/53] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:50:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:12.558519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3006a-9b41-4295-9cbe-f7c9a0f65aed None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:12.577475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 30/54] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 15:50:12.600995 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef5886-5ef9-4024-b599-0180bac649ee None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:12.657278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3006a-9b41-4295-9cbe-f7c9a0f65aed None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:12.691062 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b6a31c-4034-49a0-a243-179063020550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:12.691726 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6a31c-4034-49a0-a243-179063020550 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:12.692141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6a31c-4034-49a0-a243-179063020550 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:12.692425 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6a31c-4034-49a0-a243-179063020550 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:12.693267 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b6a31c-4034-49a0-a243-179063020550 None admin] RBAC: Adding query filter params (role.id=475521b8a0174037b162f92022aea1a1, scope.domain.id=default, user.id=bf570352584f4974b4b425f3e760bf0b) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:12.693603 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b6a31c-4034-49a0-a243-179063020550 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:12.695066 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b6a31c-4034-49a0-a243-179063020550 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:12.699993 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef5886-5ef9-4024-b599-0180bac649ee None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:12.704238 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 25/55] 10.222.0.52 () {56 vars in 1322 bytes} [Thu Jul 21 15:50:12 2022] GET /identity/v3/role_assignments?user.id=bf570352584f4974b4b425f3e760bf0b&scope.domain.id=default&role.id=475521b8a0174037b162f92022aea1a1 => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:50:12.738198 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774fbc97-7348-43fa-80ea-e40ae74dbe0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:12.739556 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fbc97-7348-43fa-80ea-e40ae74dbe0e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:12.739845 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fbc97-7348-43fa-80ea-e40ae74dbe0e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:12.740124 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fbc97-7348-43fa-80ea-e40ae74dbe0e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:12.741472 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774fbc97-7348-43fa-80ea-e40ae74dbe0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:12.742971 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774fbc97-7348-43fa-80ea-e40ae74dbe0e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:12.744671 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 31/56] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:50:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:50:12.750615 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b6a31c-4034-49a0-a243-179063020550 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:12.800378 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:12.859744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b6a31c-4034-49a0-a243-179063020550 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:12.897118 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd187e-2fbc-4092-876b-c6fdeda12ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:12.897849 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd187e-2fbc-4092-876b-c6fdeda12ac4 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:12.898137 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd187e-2fbc-4092-876b-c6fdeda12ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:12.898421 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd187e-2fbc-4092-876b-c6fdeda12ac4 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:12.899152 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd187e-2fbc-4092-876b-c6fdeda12ac4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:12.900614 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd187e-2fbc-4092-876b-c6fdeda12ac4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:12.925906 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 26/57] 10.222.0.52 () {60 vars in 1140 bytes} [Thu Jul 21 15:50:12 2022] POST /identity/v3/projects => generated 298 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:50:14.952365 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e56082-df52-457e-b2bb-d9a5fe389117 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:14.953052 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e56082-df52-457e-b2bb-d9a5fe389117 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:14.953052 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e56082-df52-457e-b2bb-d9a5fe389117 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:14.954369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 32/58] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:14.960138 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf4ad8-aa44-4809-b8f2-845596ca349d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:14.960573 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf4ad8-aa44-4809-b8f2-845596ca349d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:14.960705 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf4ad8-aa44-4809-b8f2-845596ca349d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:14.977591 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5edf4ad8-aa44-4809-b8f2-845596ca349d None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:14.978829 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5edf4ad8-aa44-4809-b8f2-845596ca349d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:15.048189 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 27/59] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:15.072456 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e56082-df52-457e-b2bb-d9a5fe389117 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:15.106966 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:15.188784 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e56082-df52-457e-b2bb-d9a5fe389117 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:15.225279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe893eae-9189-4ba5-941d-45d08a765621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:15.226110 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe893eae-9189-4ba5-941d-45d08a765621 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:15.226398 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe893eae-9189-4ba5-941d-45d08a765621 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:15.226676 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe893eae-9189-4ba5-941d-45d08a765621 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:15.227896 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe893eae-9189-4ba5-941d-45d08a765621 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:15.229359 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe893eae-9189-4ba5-941d-45d08a765621 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:15.231037 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 33/60] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:50:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:15.236847 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf4ad8-aa44-4809-b8f2-845596ca349d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:15.272841 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:15.348503 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf4ad8-aa44-4809-b8f2-845596ca349d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:15.383614 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ebc0d-e81a-479e-a1de-3268b448795b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:15.384289 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ebc0d-e81a-479e-a1de-3268b448795b None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:15.384574 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ebc0d-e81a-479e-a1de-3268b448795b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:15.384851 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ebc0d-e81a-479e-a1de-3268b448795b None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:15.385609 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ebc0d-e81a-479e-a1de-3268b448795b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:15.387069 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ebc0d-e81a-479e-a1de-3268b448795b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:15.413034 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 28/61] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 15:50:15 2022] POST /identity/v3/users => generated 275 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:50:18.658502 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b8201-a732-4d92-9ce1-0769758da2e4 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:18.658989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b8201-a732-4d92-9ce1-0769758da2e4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:18.658989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b8201-a732-4d92-9ce1-0769758da2e4 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:18.660536 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 34/62] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:18.675221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c92a8f-20b4-43dc-b887-f728a1cd2397 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:18.675745 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c92a8f-20b4-43dc-b887-f728a1cd2397 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:18.676062 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c92a8f-20b4-43dc-b887-f728a1cd2397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:18.702628 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f0c92a8f-20b4-43dc-b887-f728a1cd2397 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:18.703871 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f0c92a8f-20b4-43dc-b887-f728a1cd2397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:18.796248 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 29/63] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:18.821043 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbf1c0-567f-4cdc-a591-4eb350c756a5 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:18.821343 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbf1c0-567f-4cdc-a591-4eb350c756a5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:18.821628 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbf1c0-567f-4cdc-a591-4eb350c756a5 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:18.823024 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 35/64] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 15:50:18.831142 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626640b-f1af-4472-a884-e3366bf90b16 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:18.831435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626640b-f1af-4472-a884-e3366bf90b16 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:18.831724 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626640b-f1af-4472-a884-e3366bf90b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:18.854695 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4626640b-f1af-4472-a884-e3366bf90b16 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:18.855917 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4626640b-f1af-4472-a884-e3366bf90b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:18.864793 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:18.870873 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dbf1c0-567f-4cdc-a591-4eb350c756a5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:18.990616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 30/65] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:19.025375 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626640b-f1af-4472-a884-e3366bf90b16 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:19.040664 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:19.052812 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dbf1c0-567f-4cdc-a591-4eb350c756a5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:19.107286 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d818665c-e5a4-4fb3-955c-b907a9ce4174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:19.109708 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818665c-e5a4-4fb3-955c-b907a9ce4174 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:19.110331 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818665c-e5a4-4fb3-955c-b907a9ce4174 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:19.110918 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818665c-e5a4-4fb3-955c-b907a9ce4174 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:19.131445 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d818665c-e5a4-4fb3-955c-b907a9ce4174 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:19.143370 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d818665c-e5a4-4fb3-955c-b907a9ce4174 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:50:19.143370 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:19.143370 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:19.143370 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:19.143370 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:19.143370 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:19.143370 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:19.143370 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:19.143370 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:19.143370 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:19.143370 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:19.143370 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:19.143370 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:19.143370 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:19.143370 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:19.143370 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:19.144506 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:19.144506 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:19.144506 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:19.144506 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:19.144506 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:19.144506 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:19.144506 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:19.144506 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:19.144506 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:19.144506 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:19.144506 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:19.144506 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:19.144506 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:19.144506 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:19.144506 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:19.144506 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:19.145476 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:19.145476 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:19.145476 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:19.145476 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:19.145476 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:19.145476 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:19.145476 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:19.145476 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:19.145476 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:19.145476 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:19.145476 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:50:19.145476 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:50:19.151525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 36/66] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:50:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:19.151525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dafa381-4768-4a8f-9265-fd0b31260a74 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:19.151525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dafa381-4768-4a8f-9265-fd0b31260a74 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:19.151525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dafa381-4768-4a8f-9265-fd0b31260a74 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:19.153726 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 37/67] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:19.154998 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dafa381-4768-4a8f-9265-fd0b31260a74 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:19.164293 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626640b-f1af-4472-a884-e3366bf90b16 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:19.225547 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642613f1-4174-41ca-acae-99de6d03ac90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:19.226759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642613f1-4174-41ca-acae-99de6d03ac90 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:19.227831 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642613f1-4174-41ca-acae-99de6d03ac90 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:19.228327 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642613f1-4174-41ca-acae-99de6d03ac90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:19.257708 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642613f1-4174-41ca-acae-99de6d03ac90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:19.270024 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-642613f1-4174-41ca-acae-99de6d03ac90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:50:19.270024 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:19.270024 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:19.270024 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:19.270024 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:19.270024 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:19.270024 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:19.270024 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:19.270024 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:19.270024 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:19.270024 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:19.270024 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:19.270024 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:19.270024 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:19.270024 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:19.270024 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:19.271105 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:19.271105 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:19.271105 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:19.271105 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:19.271105 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:19.271105 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:19.271105 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:19.271105 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:19.271105 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:19.271105 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:19.271105 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:19.271105 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:19.271105 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:19.271105 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:19.271105 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:19.271105 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:19.271922 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:19.271922 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:19.271922 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:19.271922 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:19.271922 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:19.271922 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:19.271922 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:19.271922 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:19.271922 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:19.271922 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:19.271922 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:50:19.271922 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:50:19.275082 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 31/68] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:50:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:50:19.276773 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba745fb0-746f-429c-b6d5-68bb74d481b2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:19.277269 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba745fb0-746f-429c-b6d5-68bb74d481b2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:19.277742 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba745fb0-746f-429c-b6d5-68bb74d481b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:19.283875 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:19.300522 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ba745fb0-746f-429c-b6d5-68bb74d481b2 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:19.302444 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ba745fb0-746f-429c-b6d5-68bb74d481b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:19.323023 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dafa381-4768-4a8f-9265-fd0b31260a74 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:19.361804 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d85429-4d5a-45bb-85fc-5cc6103cb710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:19.362858 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d85429-4d5a-45bb-85fc-5cc6103cb710 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:19.363408 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d85429-4d5a-45bb-85fc-5cc6103cb710 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:19.364463 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d85429-4d5a-45bb-85fc-5cc6103cb710 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:19.365470 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d85429-4d5a-45bb-85fc-5cc6103cb710 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:19.366077 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d85429-4d5a-45bb-85fc-5cc6103cb710 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:19.367894 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d85429-4d5a-45bb-85fc-5cc6103cb710 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:19.383606 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 38/69] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 15:50:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:19.384924 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d85429-4d5a-45bb-85fc-5cc6103cb710 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:19.386021 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 32/70] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:19.387073 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba745fb0-746f-429c-b6d5-68bb74d481b2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:19.411771 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:19.507260 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba745fb0-746f-429c-b6d5-68bb74d481b2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:19.512166 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d85429-4d5a-45bb-85fc-5cc6103cb710 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:19.537791 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:19.543197 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f9ce2-e4cd-4f8b-aab7-c6f2bea1f74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:19.544091 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f9ce2-e4cd-4f8b-aab7-c6f2bea1f74f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:19.544378 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f9ce2-e4cd-4f8b-aab7-c6f2bea1f74f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:19.544658 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f9ce2-e4cd-4f8b-aab7-c6f2bea1f74f None admin] PATH_INFO: `/v3/users/7c7ec8ec280649a8bf1f51745c94153b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:19.548139 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a071c7be-bfcc-4a92-a372-43506d41d9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:19.548816 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071c7be-bfcc-4a92-a372-43506d41d9ad None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:19.549108 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071c7be-bfcc-4a92-a372-43506d41d9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:19.549392 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071c7be-bfcc-4a92-a372-43506d41d9ad None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:19.555971 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a071c7be-bfcc-4a92-a372-43506d41d9ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:19.555971 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a071c7be-bfcc-4a92-a372-43506d41d9ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:19.555971 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a071c7be-bfcc-4a92-a372-43506d41d9ad None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:19.561605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 39/71] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:50:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:19.562772 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ab9e7-8f79-4759-a981-0ec85d1dbbff None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:19.563071 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ab9e7-8f79-4759-a981-0ec85d1dbbff None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:19.563363 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ab9e7-8f79-4759-a981-0ec85d1dbbff None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:19.565122 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 40/72] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:19.566555 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ab9e7-8f79-4759-a981-0ec85d1dbbff None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:19.579385 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f9ce2-e4cd-4f8b-aab7-c6f2bea1f74f None admin] RBAC: Authorizing `identity:get_user(user_id=7c7ec8ec280649a8bf1f51745c94153b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:19.581338 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f9ce2-e4cd-4f8b-aab7-c6f2bea1f74f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:19.583336 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 33/73] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:19 2022] GET /identity/v3/users/7c7ec8ec280649a8bf1f51745c94153b => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:19.585250 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c1994-0da1-41ed-95fd-62c671ed5dba None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:19.585713 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c1994-0da1-41ed-95fd-62c671ed5dba None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:19.586179 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c1994-0da1-41ed-95fd-62c671ed5dba None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:19.588422 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 34/74] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:19.589410 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c1994-0da1-41ed-95fd-62c671ed5dba None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:19.664012 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:19.697461 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ab9e7-8f79-4759-a981-0ec85d1dbbff None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:19.711475 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c1994-0da1-41ed-95fd-62c671ed5dba None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:19.735921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf6170e-6379-4e6f-9d12-50b04d95c4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:19.736750 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6170e-6379-4e6f-9d12-50b04d95c4ec None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:19.737044 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6170e-6379-4e6f-9d12-50b04d95c4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:19.737332 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6170e-6379-4e6f-9d12-50b04d95c4ec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:19.751592 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf6170e-6379-4e6f-9d12-50b04d95c4ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:19.752602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78df9f0-d515-4f47-bf04-ad4fca90a113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:19.753481 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78df9f0-d515-4f47-bf04-ad4fca90a113 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:19.753798 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78df9f0-d515-4f47-bf04-ad4fca90a113 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:19.754121 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78df9f0-d515-4f47-bf04-ad4fca90a113 None admin] PATH_INFO: `/v3/users/bf570352584f4974b4b425f3e760bf0b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:19.755338 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78df9f0-d515-4f47-bf04-ad4fca90a113 None admin] RBAC: Authorizing `identity:get_user(user_id=bf570352584f4974b4b425f3e760bf0b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:19.756822 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78df9f0-d515-4f47-bf04-ad4fca90a113 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:19.758516 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 35/75] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:19 2022] GET /identity/v3/users/bf570352584f4974b4b425f3e760bf0b => generated 247 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:19.759822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-9bf6170e-6379-4e6f-9d12-50b04d95c4ec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 15:50:19.759822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:19.759822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:19.759822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:19.759822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:19.759822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:19.759822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:19.759822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:19.759822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:19.759822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:19.759822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:19.759822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:19.759822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:19.759822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:19.759822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:19.759822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:19.761076 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:19.761076 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:19.761076 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:19.761076 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:19.761076 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:19.761076 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:19.761076 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:19.761076 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:19.761076 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:19.761076 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:19.761076 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:19.761076 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:19.761076 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:19.761076 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:19.761076 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:19.761076 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:19.762510 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:19.762510 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:19.762510 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:19.762510 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:19.762510 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:19.762510 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:19.762510 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:19.762510 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:19.762510 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:19.762510 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:19.762510 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 15:50:19.762510 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:50:19.763150 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda5b90-d0cf-43cb-8863-d5c2e814839e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:19.763150 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda5b90-d0cf-43cb-8863-d5c2e814839e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:19.763150 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda5b90-d0cf-43cb-8863-d5c2e814839e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:19.763336 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 41/76] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 15:50:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:19.763336 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf6170e-6379-4e6f-9d12-50b04d95c4ec None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:19.780212 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ecda5b90-d0cf-43cb-8863-d5c2e814839e None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:19.781467 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ecda5b90-d0cf-43cb-8863-d5c2e814839e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:19.840939 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:19.855570 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 36/77] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:19.856772 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501324ff-343b-4165-a14e-6693a48d3b93 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:19.857065 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501324ff-343b-4165-a14e-6693a48d3b93 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:19.857350 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501324ff-343b-4165-a14e-6693a48d3b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:19.875320 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-501324ff-343b-4165-a14e-6693a48d3b93 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:19.876518 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-501324ff-343b-4165-a14e-6693a48d3b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:19.886994 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf6170e-6379-4e6f-9d12-50b04d95c4ec None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:19.922835 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497446b-27b1-4f06-9c2e-25af60bdf75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:19.923498 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497446b-27b1-4f06-9c2e-25af60bdf75d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:19.923786 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497446b-27b1-4f06-9c2e-25af60bdf75d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:19.924070 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497446b-27b1-4f06-9c2e-25af60bdf75d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:19.935433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:19.945080 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 37/78] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:19.945975 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501324ff-343b-4165-a14e-6693a48d3b93 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:20.019551 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a497446b-27b1-4f06-9c2e-25af60bdf75d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:20.021682 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a497446b-27b1-4f06-9c2e-25af60bdf75d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:20.058704 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501324ff-343b-4165-a14e-6693a48d3b93 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:20.078822 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:20.097866 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4231f79b-8f25-4a4e-8c62-6c6a9f231770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:20.098533 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231f79b-8f25-4a4e-8c62-6c6a9f231770 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:20.099063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231f79b-8f25-4a4e-8c62-6c6a9f231770 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:20.099356 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231f79b-8f25-4a4e-8c62-6c6a9f231770 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:20.133472 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 42/79] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:50:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 15:50:20.134494 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497446b-27b1-4f06-9c2e-25af60bdf75d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:20.180366 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:20.181359 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4231f79b-8f25-4a4e-8c62-6c6a9f231770 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:20.183061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4231f79b-8f25-4a4e-8c62-6c6a9f231770 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:20.260206 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497446b-27b1-4f06-9c2e-25af60bdf75d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:20.296046 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c921857-9eaf-4833-bb91-a38f2b9fe334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:20.296632 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c921857-9eaf-4833-bb91-a38f2b9fe334 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:20.296920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c921857-9eaf-4833-bb91-a38f2b9fe334 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:20.297193 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c921857-9eaf-4833-bb91-a38f2b9fe334 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:20.297992 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c921857-9eaf-4833-bb91-a38f2b9fe334 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:20.298301 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c921857-9eaf-4833-bb91-a38f2b9fe334 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:20.299710 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c921857-9eaf-4833-bb91-a38f2b9fe334 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:20.303911 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 38/80] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:50:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 15:50:20.304766 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4231f79b-8f25-4a4e-8c62-6c6a9f231770 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:20.309150 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 43/81] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:50:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:20.309931 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c921857-9eaf-4833-bb91-a38f2b9fe334 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:20.323782 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:20.415534 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c921857-9eaf-4833-bb91-a38f2b9fe334 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:20.417129 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:20.418091 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4231f79b-8f25-4a4e-8c62-6c6a9f231770 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:20.454179 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498e6ad-563c-492e-bfba-24792d051d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:20.454982 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498e6ad-563c-492e-bfba-24792d051d07 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:20.455270 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498e6ad-563c-492e-bfba-24792d051d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:20.455552 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498e6ad-563c-492e-bfba-24792d051d07 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:20.455638 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa71a23-51cc-4591-870e-1900a76803c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:20.456407 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa71a23-51cc-4591-870e-1900a76803c3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:20.456685 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa71a23-51cc-4591-870e-1900a76803c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:20.457019 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa71a23-51cc-4591-870e-1900a76803c3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:20.457908 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498e6ad-563c-492e-bfba-24792d051d07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:20.459299 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498e6ad-563c-492e-bfba-24792d051d07 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:20.460962 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 39/82] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:50:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:20.461778 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498e6ad-563c-492e-bfba-24792d051d07 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:20.469661 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa71a23-51cc-4591-870e-1900a76803c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:20.477857 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-2aa71a23-51cc-4591-870e-1900a76803c3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:50:20.477857 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:20.477857 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:20.477857 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:20.477857 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:20.477857 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:20.477857 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:20.477857 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:20.477857 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:20.477857 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:20.477857 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:20.477857 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:20.477857 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:20.477857 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:20.477857 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:20.477857 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:20.478775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:20.478775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:20.478775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:20.478775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:20.478775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:20.478775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:20.478775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:20.478775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:20.478775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:20.478775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:20.478775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:20.478775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:20.478775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:20.478775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:20.478775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:20.478775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:20.479690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:20.479690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:20.479690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:20.479690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:20.479690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:20.479690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:20.479690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:20.479690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:20.479690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:20.479690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:20.479690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:50:20.479690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:50:20.479690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 44/83] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:50:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:20.480586 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa71a23-51cc-4591-870e-1900a76803c3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:20.557867 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:20.574840 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498e6ad-563c-492e-bfba-24792d051d07 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:20.587809 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa71a23-51cc-4591-870e-1900a76803c3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:20.621093 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611220a9-c002-476e-b2d6-5649ded44b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:20.622083 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611220a9-c002-476e-b2d6-5649ded44b04 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:20.622375 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611220a9-c002-476e-b2d6-5649ded44b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:20.622651 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611220a9-c002-476e-b2d6-5649ded44b04 None admin] PATH_INFO: `/v3/projects/8edda125d8bb41498eca65801bd10882` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:20.623944 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611220a9-c002-476e-b2d6-5649ded44b04 None admin] RBAC: Authorizing `identity:get_project(project_id=8edda125d8bb41498eca65801bd10882)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:20.625415 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611220a9-c002-476e-b2d6-5649ded44b04 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:20.627352 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 40/84] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:50:20 2022] GET /identity/v3/projects/8edda125d8bb41498eca65801bd10882 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:20.627943 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611220a9-c002-476e-b2d6-5649ded44b04 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:20.635085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e0583-ac32-417a-a99e-caa78d313eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:20.635973 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e0583-ac32-417a-a99e-caa78d313eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:20.636260 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e0583-ac32-417a-a99e-caa78d313eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:20.636543 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e0583-ac32-417a-a99e-caa78d313eb4 None admin] PATH_INFO: `/v3/projects/8edda125d8bb41498eca65801bd10882` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:20.638643 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973e0583-ac32-417a-a99e-caa78d313eb4 None admin] RBAC: Authorizing `identity:get_project(project_id=8edda125d8bb41498eca65801bd10882)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:20.639286 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973e0583-ac32-417a-a99e-caa78d313eb4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:20.641052 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 45/85] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:50:20 2022] GET /identity/v3/projects/8edda125d8bb41498eca65801bd10882 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:20.642003 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e0583-ac32-417a-a99e-caa78d313eb4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:20.680268 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:20.751962 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611220a9-c002-476e-b2d6-5649ded44b04 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:20.758424 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e0583-ac32-417a-a99e-caa78d313eb4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:20.790870 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146581e-b8a3-4623-bf6a-907359adabdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:20.791754 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146581e-b8a3-4623-bf6a-907359adabdb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:20.792039 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146581e-b8a3-4623-bf6a-907359adabdb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:20.792318 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146581e-b8a3-4623-bf6a-907359adabdb None admin] PATH_INFO: `/v3/users/7c7ec8ec280649a8bf1f51745c94153b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:20.793566 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c146581e-b8a3-4623-bf6a-907359adabdb None admin] RBAC: Authorizing `identity:get_user(user_id=7c7ec8ec280649a8bf1f51745c94153b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:20.795176 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c146581e-b8a3-4623-bf6a-907359adabdb None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:20.796854 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 41/86] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:20 2022] GET /identity/v3/users/7c7ec8ec280649a8bf1f51745c94153b => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:20.797651 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146581e-b8a3-4623-bf6a-907359adabdb None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:20.798641 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a515658-d5c9-488d-9550-764d33937677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:20.799324 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a515658-d5c9-488d-9550-764d33937677 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:20.799674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a515658-d5c9-488d-9550-764d33937677 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:20.799889 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a515658-d5c9-488d-9550-764d33937677 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:20.800636 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a515658-d5c9-488d-9550-764d33937677 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:20.802815 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a515658-d5c9-488d-9550-764d33937677 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:20.832341 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:20.832621 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 46/87] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 15:50:20 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:50:20.832621 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a515658-d5c9-488d-9550-764d33937677 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:20.954741 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146581e-b8a3-4623-bf6a-907359adabdb None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:20.956944 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:20.977461 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a515658-d5c9-488d-9550-764d33937677 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:21.020967 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24de1ce-a68a-4c82-b4ca-01d9b6d89be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:21.023003 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24de1ce-a68a-4c82-b4ca-01d9b6d89be7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:21.023003 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24de1ce-a68a-4c82-b4ca-01d9b6d89be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:21.023003 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24de1ce-a68a-4c82-b4ca-01d9b6d89be7 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:21.023003 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24de1ce-a68a-4c82-b4ca-01d9b6d89be7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:21.023314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24de1ce-a68a-4c82-b4ca-01d9b6d89be7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:21.024676 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24de1ce-a68a-4c82-b4ca-01d9b6d89be7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:21.034153 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 42/88] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 15:50:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:21.035024 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24de1ce-a68a-4c82-b4ca-01d9b6d89be7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:21.047311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83cca1-668b-4d35-892f-300083f0e2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:21.048326 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83cca1-668b-4d35-892f-300083f0e2ff None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:21.048837 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83cca1-668b-4d35-892f-300083f0e2ff None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:21.049335 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83cca1-668b-4d35-892f-300083f0e2ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:21.051440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c83cca1-668b-4d35-892f-300083f0e2ff None admin] RBAC: Adding query filter params (role.id=54195e660fc349978b39d5794765a095, scope.project.id=8edda125d8bb41498eca65801bd10882, user.id=7c7ec8ec280649a8bf1f51745c94153b) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:21.052340 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c83cca1-668b-4d35-892f-300083f0e2ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:21.055860 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c83cca1-668b-4d35-892f-300083f0e2ff None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:21.064880 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 47/89] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:50:20 2022] GET /identity/v3/role_assignments?user.id=7c7ec8ec280649a8bf1f51745c94153b&scope.project.id=8edda125d8bb41498eca65801bd10882&role.id=54195e660fc349978b39d5794765a095 => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:21.064880 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83cca1-668b-4d35-892f-300083f0e2ff None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:21.163576 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:21.193547 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24de1ce-a68a-4c82-b4ca-01d9b6d89be7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:21.239948 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83cca1-668b-4d35-892f-300083f0e2ff None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:21.254227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af5f819-4fb4-454f-8b43-dd114937e33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:21.255723 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5f819-4fb4-454f-8b43-dd114937e33c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:21.256209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5f819-4fb4-454f-8b43-dd114937e33c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:21.256686 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5f819-4fb4-454f-8b43-dd114937e33c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:21.258521 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af5f819-4fb4-454f-8b43-dd114937e33c None admin] RBAC: Adding query filter params (role.id=475521b8a0174037b162f92022aea1a1, scope.project.id=8edda125d8bb41498eca65801bd10882, user.id=bf570352584f4974b4b425f3e760bf0b) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:21.259081 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af5f819-4fb4-454f-8b43-dd114937e33c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:21.262662 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af5f819-4fb4-454f-8b43-dd114937e33c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:21.272227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 43/90] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:50:21 2022] GET /identity/v3/role_assignments?user.id=bf570352584f4974b4b425f3e760bf0b&scope.project.id=8edda125d8bb41498eca65801bd10882&role.id=475521b8a0174037b162f92022aea1a1 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:21.273516 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af5f819-4fb4-454f-8b43-dd114937e33c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:21.289529 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a88c0d-6fba-4d17-aac3-d23949144914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:21.290636 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a88c0d-6fba-4d17-aac3-d23949144914 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:21.291185 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a88c0d-6fba-4d17-aac3-d23949144914 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:21.291678 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a88c0d-6fba-4d17-aac3-d23949144914 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:21.322427 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:21.453878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af5f819-4fb4-454f-8b43-dd114937e33c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:21.483980 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a88c0d-6fba-4d17-aac3-d23949144914 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:21.489608 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a88c0d-6fba-4d17-aac3-d23949144914 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:21.538708 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375977cc-a57e-4029-bbf4-e24c79a31598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:21.539611 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375977cc-a57e-4029-bbf4-e24c79a31598 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:21.539905 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375977cc-a57e-4029-bbf4-e24c79a31598 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:21.540189 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375977cc-a57e-4029-bbf4-e24c79a31598 None admin] PATH_INFO: `/v3/users/7c7ec8ec280649a8bf1f51745c94153b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:21.541462 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375977cc-a57e-4029-bbf4-e24c79a31598 None admin] RBAC: Authorizing `identity:get_user(user_id=7c7ec8ec280649a8bf1f51745c94153b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:21.547156 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375977cc-a57e-4029-bbf4-e24c79a31598 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:21.549262 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 44/91] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:21 2022] GET /identity/v3/users/7c7ec8ec280649a8bf1f51745c94153b => generated 275 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:21.555126 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375977cc-a57e-4029-bbf4-e24c79a31598 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:21.590035 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:21.611949 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 48/92] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:50:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 15:50:21.615977 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a88c0d-6fba-4d17-aac3-d23949144914 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:21.678018 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:21.720119 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375977cc-a57e-4029-bbf4-e24c79a31598 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:21.767600 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a88c0d-6fba-4d17-aac3-d23949144914 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:21.774370 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d78b4e-3df5-45fe-86d1-52f8ebf30ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:21.775046 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d78b4e-3df5-45fe-86d1-52f8ebf30ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:21.775578 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d78b4e-3df5-45fe-86d1-52f8ebf30ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:21.775964 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d78b4e-3df5-45fe-86d1-52f8ebf30ad9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:21.861461 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d78b4e-3df5-45fe-86d1-52f8ebf30ad9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:21.862367 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0bd8c5-5126-47fd-9f1e-2b1b60879c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:21.863326 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0bd8c5-5126-47fd-9f1e-2b1b60879c45 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:21.863624 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d78b4e-3df5-45fe-86d1-52f8ebf30ad9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:21.863713 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0bd8c5-5126-47fd-9f1e-2b1b60879c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:21.863914 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0bd8c5-5126-47fd-9f1e-2b1b60879c45 None admin] PATH_INFO: `/v3/projects/8edda125d8bb41498eca65801bd10882` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:21.865258 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0bd8c5-5126-47fd-9f1e-2b1b60879c45 None admin] RBAC: Authorizing `identity:get_project(project_id=8edda125d8bb41498eca65801bd10882)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:21.866801 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0bd8c5-5126-47fd-9f1e-2b1b60879c45 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:21.868598 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 49/93] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:50:21 2022] GET /identity/v3/projects/8edda125d8bb41498eca65801bd10882 => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:50:21.876555 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0bd8c5-5126-47fd-9f1e-2b1b60879c45 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:21.909992 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:21.981399 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 45/94] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:50:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 15:50:21.987503 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d78b4e-3df5-45fe-86d1-52f8ebf30ad9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:22.110068 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d78b4e-3df5-45fe-86d1-52f8ebf30ad9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:22.130089 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:22.155486 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62069c-7273-4c7d-90a5-379f6a11e2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:22.157139 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62069c-7273-4c7d-90a5-379f6a11e2c2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:22.157500 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62069c-7273-4c7d-90a5-379f6a11e2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:22.157783 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62069c-7273-4c7d-90a5-379f6a11e2c2 None admin] PATH_INFO: `/v3/projects/82a0854bb4bb489680109c44becf6ac6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:22.159178 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a62069c-7273-4c7d-90a5-379f6a11e2c2 None admin] RBAC: Authorizing `identity:get_project(project_id=82a0854bb4bb489680109c44becf6ac6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:22.160734 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a62069c-7273-4c7d-90a5-379f6a11e2c2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:22.162517 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 46/95] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:50:21 2022] GET /identity/v3/projects/82a0854bb4bb489680109c44becf6ac6 => generated 312 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:50:22.168887 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62069c-7273-4c7d-90a5-379f6a11e2c2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:22.188184 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0bd8c5-5126-47fd-9f1e-2b1b60879c45 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:22.240311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:22.294342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c7935-a78f-4646-b741-91f626d5bcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:22.295375 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c7935-a78f-4646-b741-91f626d5bcdd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:22.295903 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c7935-a78f-4646-b741-91f626d5bcdd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:22.296375 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c7935-a78f-4646-b741-91f626d5bcdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:22.297416 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355c7935-a78f-4646-b741-91f626d5bcdd None admin] RBAC: Adding query filter params (role.id=85dcd5509fca4911881d45baf5f1d5a5, scope.project.id=8edda125d8bb41498eca65801bd10882, user.id=7c7ec8ec280649a8bf1f51745c94153b) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:22.297947 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355c7935-a78f-4646-b741-91f626d5bcdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:22.299610 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355c7935-a78f-4646-b741-91f626d5bcdd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:22.301453 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62069c-7273-4c7d-90a5-379f6a11e2c2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:22.314997 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 50/96] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:50:21 2022] GET /identity/v3/role_assignments?user.id=7c7ec8ec280649a8bf1f51745c94153b&scope.project.id=8edda125d8bb41498eca65801bd10882&role.id=85dcd5509fca4911881d45baf5f1d5a5 => generated 259 bytes in 441 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:50:22.351090 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd4adad-4c39-4ce9-85e0-205d22f0f129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:22.352289 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4adad-4c39-4ce9-85e0-205d22f0f129 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:22.352746 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4adad-4c39-4ce9-85e0-205d22f0f129 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:22.353089 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4adad-4c39-4ce9-85e0-205d22f0f129 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:22.354108 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd4adad-4c39-4ce9-85e0-205d22f0f129 None admin] RBAC: Adding query filter params (role.id=54195e660fc349978b39d5794765a095, scope.project.id=82a0854bb4bb489680109c44becf6ac6, user.id=7c7ec8ec280649a8bf1f51745c94153b) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:22.354440 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd4adad-4c39-4ce9-85e0-205d22f0f129 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:22.355981 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd4adad-4c39-4ce9-85e0-205d22f0f129 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:22.363069 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 47/97] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:50:22 2022] GET /identity/v3/role_assignments?user.id=7c7ec8ec280649a8bf1f51745c94153b&scope.project.id=82a0854bb4bb489680109c44becf6ac6&role.id=54195e660fc349978b39d5794765a095 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:50:24.265990 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825fb22-2b95-4e0b-ad44-61d26e1a31e2 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:24.266546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825fb22-2b95-4e0b-ad44-61d26e1a31e2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:24.266546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825fb22-2b95-4e0b-ad44-61d26e1a31e2 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:24.268221 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 51/98] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:24.280128 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c61e2-4761-4ee9-a7f1-da299d767641 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:24.281081 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c61e2-4761-4ee9-a7f1-da299d767641 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:24.281565 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c61e2-4761-4ee9-a7f1-da299d767641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:24.302644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-729c61e2-4761-4ee9-a7f1-da299d767641 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:24.304268 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-729c61e2-4761-4ee9-a7f1-da299d767641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:24.318575 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1013fee-1624-4477-8f3c-5bd99b9f2fa2 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:24.318893 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1013fee-1624-4477-8f3c-5bd99b9f2fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:24.319152 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1013fee-1624-4477-8f3c-5bd99b9f2fa2 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:24.320840 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 52/99] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:24.340137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bd2c3-9e36-403f-a2d1-0f0feaff802d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:24.340471 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bd2c3-9e36-403f-a2d1-0f0feaff802d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:24.340713 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bd2c3-9e36-403f-a2d1-0f0feaff802d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:24.363999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-cd5bd2c3-9e36-403f-a2d1-0f0feaff802d None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:24.365234 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cd5bd2c3-9e36-403f-a2d1-0f0feaff802d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:24.401471 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:24.450756 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 48/100] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:24.487286 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 53/101] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:24.535593 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c61e2-4761-4ee9-a7f1-da299d767641 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:24.549539 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5bd2c3-9e36-403f-a2d1-0f0feaff802d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:24.683435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c61e2-4761-4ee9-a7f1-da299d767641 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:24.695109 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5bd2c3-9e36-403f-a2d1-0f0feaff802d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:24.713513 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:24.732308 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088eb7d0-b36d-498f-8a34-47a65b486a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:24.733324 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088eb7d0-b36d-498f-8a34-47a65b486a3b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:24.733916 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088eb7d0-b36d-498f-8a34-47a65b486a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:24.734412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088eb7d0-b36d-498f-8a34-47a65b486a3b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:24.742790 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da04a337-4531-4ec6-86cd-446e83b3718c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:24.744010 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da04a337-4531-4ec6-86cd-446e83b3718c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:24.744767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da04a337-4531-4ec6-86cd-446e83b3718c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:24.745212 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da04a337-4531-4ec6-86cd-446e83b3718c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:24.748112 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088eb7d0-b36d-498f-8a34-47a65b486a3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:24.756814 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-088eb7d0-b36d-498f-8a34-47a65b486a3b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:50:24.756814 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:24.756814 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:24.756814 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:24.756814 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:24.756814 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:24.756814 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:24.756814 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:24.756814 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:24.756814 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:24.756814 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:24.756814 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:24.756814 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:24.756814 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:24.756814 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:24.756814 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:24.758047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:24.758047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:24.758047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:24.758047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:24.758047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:24.758047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:24.758047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:24.758047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:24.758047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:24.758047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:24.758047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:24.758047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:24.758047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:24.758047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:24.758047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:24.758047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:24.758863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:24.758863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:24.758863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:24.758863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:24.758863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:24.758863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:24.758863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:24.758863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:24.758863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:24.758863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:24.758863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:50:24.758863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:50:24.759481 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da04a337-4531-4ec6-86cd-446e83b3718c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:24.761618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 49/102] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:50:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:24.763158 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381819e-a962-4133-b4b2-39b9353d9a30 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:24.763669 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381819e-a962-4133-b4b2-39b9353d9a30 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:24.764159 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381819e-a962-4133-b4b2-39b9353d9a30 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:24.766153 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 50/103] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:24.767598 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d3362-fdc5-4949-9038-80c995deec88 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:24.768094 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d3362-fdc5-4949-9038-80c995deec88 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:24.768914 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d3362-fdc5-4949-9038-80c995deec88 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:24.770474 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-da04a337-4531-4ec6-86cd-446e83b3718c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:50:24.770474 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:24.770474 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:24.770474 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:24.770474 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:24.770474 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:24.770474 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:24.770474 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:24.770474 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:24.770474 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:24.770474 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:24.770474 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:24.770474 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:24.770474 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:24.770474 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:24.770474 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:24.771337 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:24.771337 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:24.771337 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:24.771337 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:24.771337 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:24.771337 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:24.771337 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:24.771337 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:24.771337 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:24.771337 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:24.771337 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:24.771337 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:24.771337 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:24.771337 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:24.771337 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:24.771337 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:24.772158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:24.772158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:24.772158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:24.772158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:24.772158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:24.772158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:24.772158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:24.772158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:24.772158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:24.772158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:24.772158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:50:24.772158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:50:24.772158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 54/104] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:50:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:24.772819 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da04a337-4531-4ec6-86cd-446e83b3718c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:24.773887 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 51/105] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:24.775516 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97543992-831f-4f65-8218-0510f5f86d6b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:24.776087 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97543992-831f-4f65-8218-0510f5f86d6b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:24.776559 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97543992-831f-4f65-8218-0510f5f86d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:24.797143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-97543992-831f-4f65-8218-0510f5f86d6b None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:24.798750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-97543992-831f-4f65-8218-0510f5f86d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:24.835011 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:24.880528 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 52/106] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:24.882361 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2ac5a-499c-4215-a9ad-e548ab3c8066 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:24.882854 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2ac5a-499c-4215-a9ad-e548ab3c8066 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:24.883329 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2ac5a-499c-4215-a9ad-e548ab3c8066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:24.902002 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-79f2ac5a-499c-4215-a9ad-e548ab3c8066 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:24.903567 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-79f2ac5a-499c-4215-a9ad-e548ab3c8066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:24.911244 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da04a337-4531-4ec6-86cd-446e83b3718c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:24.957956 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ab4ca-f545-4e29-b50e-3a01b9f4f0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:24.962449 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ab4ca-f545-4e29-b50e-3a01b9f4f0a0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:24.962865 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ab4ca-f545-4e29-b50e-3a01b9f4f0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:24.963332 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ab4ca-f545-4e29-b50e-3a01b9f4f0a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:24.964733 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ab4ca-f545-4e29-b50e-3a01b9f4f0a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:24.965105 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ab4ca-f545-4e29-b50e-3a01b9f4f0a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:24.967290 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ab4ca-f545-4e29-b50e-3a01b9f4f0a0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:24.986663 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 55/107] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 15:50:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:24.990294 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ab4ca-f545-4e29-b50e-3a01b9f4f0a0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:25.003060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 53/108] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:25.004466 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2ac5a-499c-4215-a9ad-e548ab3c8066 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:25.034571 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:25.120818 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ab4ca-f545-4e29-b50e-3a01b9f4f0a0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:25.136149 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:25.141278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2ac5a-499c-4215-a9ad-e548ab3c8066 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:25.166861 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca9487d-a6af-4b41-afc4-f71617641bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:25.167105 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9487d-a6af-4b41-afc4-f71617641bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:25.167371 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9487d-a6af-4b41-afc4-f71617641bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:25.167659 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9487d-a6af-4b41-afc4-f71617641bc3 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:25.168613 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca9487d-a6af-4b41-afc4-f71617641bc3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:25.168933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca9487d-a6af-4b41-afc4-f71617641bc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:25.170399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca9487d-a6af-4b41-afc4-f71617641bc3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:25.179857 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 56/109] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:50:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:25.180740 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca9487d-a6af-4b41-afc4-f71617641bc3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:25.185348 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6d7f7-e7c3-4161-ac4d-4d3fad809a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:25.185523 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6d7f7-e7c3-4161-ac4d-4d3fad809a42 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:25.185921 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6d7f7-e7c3-4161-ac4d-4d3fad809a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:25.186201 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6d7f7-e7c3-4161-ac4d-4d3fad809a42 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:25.201533 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd6d7f7-e7c3-4161-ac4d-4d3fad809a42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:25.210854 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-3bd6d7f7-e7c3-4161-ac4d-4d3fad809a42 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 15:50:25.210854 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:25.210854 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:25.210854 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:25.210854 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:25.210854 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:25.210854 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:25.210854 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:25.210854 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:25.210854 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:25.210854 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:25.210854 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:25.210854 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:25.210854 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:25.210854 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:25.210854 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:25.211881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:25.211881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:25.211881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:25.211881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:25.211881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:25.211881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:25.211881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:25.211881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:25.211881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:25.211881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:25.211881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:25.211881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:25.211881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:25.211881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:25.211881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:25.211881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:25.212802 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:25.212802 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:25.212802 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:25.212802 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:25.212802 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:25.212802 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:25.212802 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:25.212802 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:25.212802 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:25.212802 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:25.212802 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 15:50:25.212802 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:50:25.212802 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 54/110] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:50:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:25.213537 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6d7f7-e7c3-4161-ac4d-4d3fad809a42 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:25.313730 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:25.321302 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca9487d-a6af-4b41-afc4-f71617641bc3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:25.338124 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6d7f7-e7c3-4161-ac4d-4d3fad809a42 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:25.360035 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6662a0c-1947-4396-8598-ec5672b211fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:25.360914 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6662a0c-1947-4396-8598-ec5672b211fd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:25.361200 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6662a0c-1947-4396-8598-ec5672b211fd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:25.361479 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6662a0c-1947-4396-8598-ec5672b211fd None admin] PATH_INFO: `/v3/users/7c7ec8ec280649a8bf1f51745c94153b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:25.362733 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6662a0c-1947-4396-8598-ec5672b211fd None admin] RBAC: Authorizing `identity:get_user(user_id=7c7ec8ec280649a8bf1f51745c94153b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:25.364258 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6662a0c-1947-4396-8598-ec5672b211fd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:25.365959 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 57/111] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:25 2022] GET /identity/v3/users/7c7ec8ec280649a8bf1f51745c94153b => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:25.365959 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54dc886-b8ee-490f-a7a4-39628a8327a6 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:25.365959 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54dc886-b8ee-490f-a7a4-39628a8327a6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:25.365959 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54dc886-b8ee-490f-a7a4-39628a8327a6 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:25.365959 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 58/112] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:25.365959 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54dc886-b8ee-490f-a7a4-39628a8327a6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:25.381476 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec14be8-9009-4dad-8eb2-fb90285578b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:25.382356 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec14be8-9009-4dad-8eb2-fb90285578b4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:25.382642 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec14be8-9009-4dad-8eb2-fb90285578b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:25.382975 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec14be8-9009-4dad-8eb2-fb90285578b4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:25.384163 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec14be8-9009-4dad-8eb2-fb90285578b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:25.385651 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec14be8-9009-4dad-8eb2-fb90285578b4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:25.387433 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 55/113] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:50:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:25.388232 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec14be8-9009-4dad-8eb2-fb90285578b4 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:25.418917 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:25.483099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54dc886-b8ee-490f-a7a4-39628a8327a6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:25.499283 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec14be8-9009-4dad-8eb2-fb90285578b4 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:25.518752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a9171-dfe6-4683-b6eb-ac24544253a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:25.520278 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a9171-dfe6-4683-b6eb-ac24544253a6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:25.520570 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a9171-dfe6-4683-b6eb-ac24544253a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:25.520853 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a9171-dfe6-4683-b6eb-ac24544253a6 None admin] PATH_INFO: `/v3/users/bf570352584f4974b4b425f3e760bf0b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:25.522114 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a9171-dfe6-4683-b6eb-ac24544253a6 None admin] RBAC: Authorizing `identity:get_user(user_id=bf570352584f4974b4b425f3e760bf0b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:25.523601 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a9171-dfe6-4683-b6eb-ac24544253a6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:25.525295 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 59/114] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:25 2022] GET /identity/v3/users/bf570352584f4974b4b425f3e760bf0b => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:25.526633 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd437af-e379-4087-9749-fab5cd15fe65 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:25.526924 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd437af-e379-4087-9749-fab5cd15fe65 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:25.527211 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd437af-e379-4087-9749-fab5cd15fe65 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:25.528479 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 60/115] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:25.529599 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a4d14-5518-404f-9322-8bdeed645e05 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:25.529962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a4d14-5518-404f-9322-8bdeed645e05 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:25.530232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a4d14-5518-404f-9322-8bdeed645e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:25.539461 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2402653b-a664-45bd-9beb-9cf7c24fb2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:25.540135 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402653b-a664-45bd-9beb-9cf7c24fb2c5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:25.540422 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402653b-a664-45bd-9beb-9cf7c24fb2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:25.540705 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402653b-a664-45bd-9beb-9cf7c24fb2c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:25.541468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2402653b-a664-45bd-9beb-9cf7c24fb2c5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:25.541812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2402653b-a664-45bd-9beb-9cf7c24fb2c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:25.543207 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2402653b-a664-45bd-9beb-9cf7c24fb2c5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:25.547034 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-771a4d14-5518-404f-9322-8bdeed645e05 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:25.548274 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-771a4d14-5518-404f-9322-8bdeed645e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:25.552485 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 56/116] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 15:50:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:25.553029 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2402653b-a664-45bd-9beb-9cf7c24fb2c5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:25.577933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:25.623559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 61/117] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:25.624329 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a4d14-5518-404f-9322-8bdeed645e05 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:25.668802 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:25.682603 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2402653b-a664-45bd-9beb-9cf7c24fb2c5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:25.724622 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7329cca5-af65-43b7-a543-a3a563c8250b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:25.725818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329cca5-af65-43b7-a543-a3a563c8250b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:25.726108 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329cca5-af65-43b7-a543-a3a563c8250b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:25.726388 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329cca5-af65-43b7-a543-a3a563c8250b None admin] PATH_INFO: `/v3/projects/8edda125d8bb41498eca65801bd10882` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:25.728048 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7329cca5-af65-43b7-a543-a3a563c8250b None admin] RBAC: Authorizing `identity:get_project(project_id=8edda125d8bb41498eca65801bd10882)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:25.729537 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7329cca5-af65-43b7-a543-a3a563c8250b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:25.736091 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 57/118] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:50:25 2022] GET /identity/v3/projects/8edda125d8bb41498eca65801bd10882 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:25.736091 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7329cca5-af65-43b7-a543-a3a563c8250b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:25.742428 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a4d14-5518-404f-9322-8bdeed645e05 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:25.781854 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4208e4-6784-418b-acd9-1b8b187f69a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:25.782525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4208e4-6784-418b-acd9-1b8b187f69a1 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:25.782813 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4208e4-6784-418b-acd9-1b8b187f69a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:25.783094 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4208e4-6784-418b-acd9-1b8b187f69a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:25.783818 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4208e4-6784-418b-acd9-1b8b187f69a1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:25.785321 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4208e4-6784-418b-acd9-1b8b187f69a1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:25.816927 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 62/119] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 21 15:50:25 2022] POST /identity/v3/projects => generated 302 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:50:25.819329 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40688d-00be-4853-a738-3b7a9c63b0b6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:25.819701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40688d-00be-4853-a738-3b7a9c63b0b6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:25.820170 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40688d-00be-4853-a738-3b7a9c63b0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:25.840905 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-cd40688d-00be-4853-a738-3b7a9c63b0b6 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:25.842119 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cd40688d-00be-4853-a738-3b7a9c63b0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:25.859747 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:25.875401 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7329cca5-af65-43b7-a543-a3a563c8250b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:25.922363 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78051f2b-a9d3-47d2-a333-2af5dd4cd4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:25.923538 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78051f2b-a9d3-47d2-a333-2af5dd4cd4b4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:25.923964 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78051f2b-a9d3-47d2-a333-2af5dd4cd4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:25.924303 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78051f2b-a9d3-47d2-a333-2af5dd4cd4b4 None admin] PATH_INFO: `/v3/projects/8edda125d8bb41498eca65801bd10882` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:25.926168 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78051f2b-a9d3-47d2-a333-2af5dd4cd4b4 None admin] RBAC: Authorizing `identity:get_project(project_id=8edda125d8bb41498eca65801bd10882)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:25.927935 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78051f2b-a9d3-47d2-a333-2af5dd4cd4b4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:25.934746 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 58/120] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:50:25 2022] GET /identity/v3/projects/8edda125d8bb41498eca65801bd10882 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:25.934746 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78051f2b-a9d3-47d2-a333-2af5dd4cd4b4 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:25.950261 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 63/121] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:25.950261 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40688d-00be-4853-a738-3b7a9c63b0b6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:25.969069 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:26.069502 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78051f2b-a9d3-47d2-a333-2af5dd4cd4b4 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:26.071991 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40688d-00be-4853-a738-3b7a9c63b0b6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:26.111318 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd136b-786d-430b-a970-600aa4089bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:26.112110 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd136b-786d-430b-a970-600aa4089bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:26.112407 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd136b-786d-430b-a970-600aa4089bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:26.112685 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd136b-786d-430b-a970-600aa4089bb9 None admin] PATH_INFO: `/v3/users/3418daf276a84936bf5896c885e85a1f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:26.117945 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181dbd00-a10f-4c6b-ba8f-c6b9bcfd536e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:26.119314 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181dbd00-a10f-4c6b-ba8f-c6b9bcfd536e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:26.119826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181dbd00-a10f-4c6b-ba8f-c6b9bcfd536e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:26.120333 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181dbd00-a10f-4c6b-ba8f-c6b9bcfd536e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:26.128252 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd136b-786d-430b-a970-600aa4089bb9 None admin] RBAC: Authorizing `identity:get_user(user_id=3418daf276a84936bf5896c885e85a1f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:26.130437 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd136b-786d-430b-a970-600aa4089bb9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:26.132561 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 59/122] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:25 2022] GET /identity/v3/users/3418daf276a84936bf5896c885e85a1f => generated 289 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:26.133410 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd136b-786d-430b-a970-600aa4089bb9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:26.139429 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181dbd00-a10f-4c6b-ba8f-c6b9bcfd536e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:26.156238 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-181dbd00-a10f-4c6b-ba8f-c6b9bcfd536e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 15:50:26.156238 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:26.156238 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:26.156238 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:26.156238 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:26.156238 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:26.156238 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:26.156238 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:26.156238 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:26.156238 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:26.156238 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:26.156238 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:26.156238 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:26.156238 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:26.156238 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:26.156238 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:26.157302 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:26.157302 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:26.157302 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:26.157302 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:26.157302 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:26.157302 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:26.157302 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:26.157302 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:26.157302 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:26.157302 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:26.157302 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:26.157302 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:26.157302 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:26.157302 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:26.157302 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:26.157302 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:26.158305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:26.158305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:26.158305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:26.158305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:26.158305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:26.158305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:26.158305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:26.158305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:26.158305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:26.158305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:26.158305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 15:50:26.158305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:50:26.160410 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 64/123] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 15:50:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:26.161451 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181dbd00-a10f-4c6b-ba8f-c6b9bcfd536e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:26.168820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:26.247639 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:26.273223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd136b-786d-430b-a970-600aa4089bb9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:26.297333 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181dbd00-a10f-4c6b-ba8f-c6b9bcfd536e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:26.313918 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b0fb3-265e-4b82-91aa-91ecffe75590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:26.314916 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b0fb3-265e-4b82-91aa-91ecffe75590 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:26.315271 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b0fb3-265e-4b82-91aa-91ecffe75590 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:26.315619 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b0fb3-265e-4b82-91aa-91ecffe75590 None admin] PATH_INFO: `/v3/projects/8edda125d8bb41498eca65801bd10882/users/7c7ec8ec280649a8bf1f51745c94153b/roles/54195e660fc349978b39d5794765a095` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:26.318214 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6b0fb3-265e-4b82-91aa-91ecffe75590 None admin] RBAC: Authorizing `identity:create_grant(project_id=8edda125d8bb41498eca65801bd10882, user_id=7c7ec8ec280649a8bf1f51745c94153b, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:26.320202 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6b0fb3-265e-4b82-91aa-91ecffe75590 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:26.345611 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 60/124] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 15:50:26 2022] PUT /identity/v3/projects/8edda125d8bb41498eca65801bd10882/users/7c7ec8ec280649a8bf1f51745c94153b/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 15:50:26.345611 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b0fb3-265e-4b82-91aa-91ecffe75590 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:26.364301 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569a165-fed3-43ad-85a5-58a75e7e3687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:26.365248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569a165-fed3-43ad-85a5-58a75e7e3687 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:26.365947 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569a165-fed3-43ad-85a5-58a75e7e3687 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:26.366309 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569a165-fed3-43ad-85a5-58a75e7e3687 None admin] PATH_INFO: `/v3/projects/8edda125d8bb41498eca65801bd10882/users/bf570352584f4974b4b425f3e760bf0b/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:26.369590 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c569a165-fed3-43ad-85a5-58a75e7e3687 None admin] RBAC: Authorizing `identity:create_grant(project_id=8edda125d8bb41498eca65801bd10882, user_id=bf570352584f4974b4b425f3e760bf0b, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:26.371624 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c569a165-fed3-43ad-85a5-58a75e7e3687 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:26.394375 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 65/125] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 15:50:26 2022] PUT /identity/v3/projects/8edda125d8bb41498eca65801bd10882/users/bf570352584f4974b4b425f3e760bf0b/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 15:50:26.395657 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569a165-fed3-43ad-85a5-58a75e7e3687 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:26.492498 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:26.559695 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569a165-fed3-43ad-85a5-58a75e7e3687 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:26.581102 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b0fb3-265e-4b82-91aa-91ecffe75590 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:26.631671 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b647dc-986e-4009-a6ed-464a36a4ce79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:26.632632 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b647dc-986e-4009-a6ed-464a36a4ce79 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:26.633117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b647dc-986e-4009-a6ed-464a36a4ce79 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:26.633500 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b647dc-986e-4009-a6ed-464a36a4ce79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:26.637603 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1529225a-e800-4cae-adf7-c715a0bd8d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:26.639808 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1529225a-e800-4cae-adf7-c715a0bd8d33 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:26.640386 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1529225a-e800-4cae-adf7-c715a0bd8d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:26.640848 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1529225a-e800-4cae-adf7-c715a0bd8d33 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:26.655705 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1529225a-e800-4cae-adf7-c715a0bd8d33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:26.665911 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-1529225a-e800-4cae-adf7-c715a0bd8d33 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:50:26.665911 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:26.665911 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:26.665911 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:26.665911 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:26.665911 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:26.665911 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:26.665911 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:26.665911 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:26.665911 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:26.665911 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:26.665911 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:26.665911 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:26.665911 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:26.665911 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:26.665911 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:26.666897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:26.666897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:26.666897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:26.666897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:26.666897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:26.666897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:26.666897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:26.666897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:26.666897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:26.666897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:26.666897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:26.666897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:26.666897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:26.666897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:26.666897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:26.666897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:26.667708 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:26.667708 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:26.667708 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:26.667708 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:26.667708 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:26.667708 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:26.667708 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:26.667708 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:26.667708 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:26.667708 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:26.667708 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:50:26.667708 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:50:26.670845 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 61/126] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:50:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:26.672212 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1529225a-e800-4cae-adf7-c715a0bd8d33 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:26.677051 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:26.744155 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b647dc-986e-4009-a6ed-464a36a4ce79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:26.746716 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b647dc-986e-4009-a6ed-464a36a4ce79 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:26.807258 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1529225a-e800-4cae-adf7-c715a0bd8d33 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:26.832409 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:26.850862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c18428-bcaf-4453-94b5-ce3ebc7456ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:26.852315 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c18428-bcaf-4453-94b5-ce3ebc7456ac None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:26.852753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c18428-bcaf-4453-94b5-ce3ebc7456ac None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:26.853449 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c18428-bcaf-4453-94b5-ce3ebc7456ac None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:26.854902 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c18428-bcaf-4453-94b5-ce3ebc7456ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:26.855581 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c18428-bcaf-4453-94b5-ce3ebc7456ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:26.858292 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c18428-bcaf-4453-94b5-ce3ebc7456ac None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:26.869722 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 62/127] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:50:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:26.871453 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c18428-bcaf-4453-94b5-ce3ebc7456ac None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:26.878135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 66/128] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:50:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 15:50:26.879816 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b647dc-986e-4009-a6ed-464a36a4ce79 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:26.981969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:27.058327 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b647dc-986e-4009-a6ed-464a36a4ce79 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:27.062972 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c18428-bcaf-4453-94b5-ce3ebc7456ac None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:27.126975 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57b451-c0ef-4eba-9fe8-9c9d7a4f43f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:27.127864 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57b451-c0ef-4eba-9fe8-9c9d7a4f43f4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:27.128151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57b451-c0ef-4eba-9fe8-9c9d7a4f43f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:27.128427 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57b451-c0ef-4eba-9fe8-9c9d7a4f43f4 None admin] PATH_INFO: `/v3/users/7c7ec8ec280649a8bf1f51745c94153b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:27.129696 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fecd8e-9fe9-453e-b104-296ff150ba37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:27.130369 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fecd8e-9fe9-453e-b104-296ff150ba37 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:27.130648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fecd8e-9fe9-453e-b104-296ff150ba37 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:27.130922 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fecd8e-9fe9-453e-b104-296ff150ba37 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:27.131998 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57b451-c0ef-4eba-9fe8-9c9d7a4f43f4 None admin] RBAC: Authorizing `identity:get_user(user_id=7c7ec8ec280649a8bf1f51745c94153b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:27.133553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57b451-c0ef-4eba-9fe8-9c9d7a4f43f4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:27.135313 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 67/129] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:26 2022] GET /identity/v3/users/7c7ec8ec280649a8bf1f51745c94153b => generated 275 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:27.136090 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57b451-c0ef-4eba-9fe8-9c9d7a4f43f4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:27.137027 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fecd8e-9fe9-453e-b104-296ff150ba37 None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:27.137354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fecd8e-9fe9-453e-b104-296ff150ba37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:27.138806 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fecd8e-9fe9-453e-b104-296ff150ba37 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:27.160080 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 63/130] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 15:50:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:27.160977 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fecd8e-9fe9-453e-b104-296ff150ba37 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:27.216133 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:27.286972 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fecd8e-9fe9-453e-b104-296ff150ba37 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:27.330935 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9378106d-4f59-4924-9a0d-7a9c2fd1b0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:27.331867 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9378106d-4f59-4924-9a0d-7a9c2fd1b0f6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:27.332652 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9378106d-4f59-4924-9a0d-7a9c2fd1b0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:27.332957 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9378106d-4f59-4924-9a0d-7a9c2fd1b0f6 None admin] PATH_INFO: `/v3/projects/8edda125d8bb41498eca65801bd10882` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:27.333990 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:27.336508 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9378106d-4f59-4924-9a0d-7a9c2fd1b0f6 None admin] RBAC: Authorizing `identity:get_project(project_id=8edda125d8bb41498eca65801bd10882)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:27.339759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9378106d-4f59-4924-9a0d-7a9c2fd1b0f6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:27.344365 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 64/131] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:50:27 2022] GET /identity/v3/projects/8edda125d8bb41498eca65801bd10882 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:27.345228 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9378106d-4f59-4924-9a0d-7a9c2fd1b0f6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:27.387187 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57b451-c0ef-4eba-9fe8-9c9d7a4f43f4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:27.489417 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9378106d-4f59-4924-9a0d-7a9c2fd1b0f6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:27.526502 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64b26c-be2a-4ca7-a0d0-eb87db9626a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:27.527290 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64b26c-be2a-4ca7-a0d0-eb87db9626a0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:27.527578 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64b26c-be2a-4ca7-a0d0-eb87db9626a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:27.527858 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64b26c-be2a-4ca7-a0d0-eb87db9626a0 None admin] PATH_INFO: `/v3/projects/8edda125d8bb41498eca65801bd10882` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:27.529523 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a64b26c-be2a-4ca7-a0d0-eb87db9626a0 None admin] RBAC: Authorizing `identity:get_project(project_id=8edda125d8bb41498eca65801bd10882)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:27.531044 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a64b26c-be2a-4ca7-a0d0-eb87db9626a0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:27.534648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e410a-2577-469f-a13b-80fc2cb1cd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:27.536090 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e410a-2577-469f-a13b-80fc2cb1cd21 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:27.536376 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e410a-2577-469f-a13b-80fc2cb1cd21 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:27.536655 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e410a-2577-469f-a13b-80fc2cb1cd21 None admin] PATH_INFO: `/v3/users/7c7ec8ec280649a8bf1f51745c94153b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:27.538159 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8e410a-2577-469f-a13b-80fc2cb1cd21 None admin] RBAC: Authorizing `identity:get_user(user_id=7c7ec8ec280649a8bf1f51745c94153b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:27.539624 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8e410a-2577-469f-a13b-80fc2cb1cd21 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:27.541291 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 65/132] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:27 2022] GET /identity/v3/users/7c7ec8ec280649a8bf1f51745c94153b => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:27.545934 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 68/133] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:50:27 2022] GET /identity/v3/projects/8edda125d8bb41498eca65801bd10882 => generated 298 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:27.546900 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64b26c-be2a-4ca7-a0d0-eb87db9626a0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:27.547830 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e410a-2577-469f-a13b-80fc2cb1cd21 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:27.680904 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:27.684117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64b26c-be2a-4ca7-a0d0-eb87db9626a0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:27.705961 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:27.731420 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf2bfe-0946-4e2f-bee7-409e20253bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:27.732101 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf2bfe-0946-4e2f-bee7-409e20253bdd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:27.732397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf2bfe-0946-4e2f-bee7-409e20253bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:27.732678 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf2bfe-0946-4e2f-bee7-409e20253bdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:27.733533 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdf2bfe-0946-4e2f-bee7-409e20253bdd None admin] RBAC: Adding query filter params (role.id=f4d0b371e4b140d9a9f6d9c6983c0ec8, scope.project.id=8edda125d8bb41498eca65801bd10882, user.id=3418daf276a84936bf5896c885e85a1f) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:27.733880 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdf2bfe-0946-4e2f-bee7-409e20253bdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:27.735393 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdf2bfe-0946-4e2f-bee7-409e20253bdd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:27.751541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 69/134] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:50:27 2022] GET /identity/v3/role_assignments?user.id=3418daf276a84936bf5896c885e85a1f&scope.project.id=8edda125d8bb41498eca65801bd10882&role.id=f4d0b371e4b140d9a9f6d9c6983c0ec8 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:27.752412 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf2bfe-0946-4e2f-bee7-409e20253bdd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:27.783405 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e410a-2577-469f-a13b-80fc2cb1cd21 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:27.942952 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf2bfe-0946-4e2f-bee7-409e20253bdd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:27.962432 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54005854-3977-4ea4-bc82-8c3296ae97a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:27.963362 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54005854-3977-4ea4-bc82-8c3296ae97a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:27.963654 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54005854-3977-4ea4-bc82-8c3296ae97a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:27.964034 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54005854-3977-4ea4-bc82-8c3296ae97a4 None admin] PATH_INFO: `/v3/projects/8edda125d8bb41498eca65801bd10882/users/7c7ec8ec280649a8bf1f51745c94153b/roles/85dcd5509fca4911881d45baf5f1d5a5` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:27.974842 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54005854-3977-4ea4-bc82-8c3296ae97a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=8edda125d8bb41498eca65801bd10882, user_id=7c7ec8ec280649a8bf1f51745c94153b, role_id=85dcd5509fca4911881d45baf5f1d5a5)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:27.976356 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54005854-3977-4ea4-bc82-8c3296ae97a4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:27.983433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14972603-914b-469f-b3eb-78f29f727bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:27.984496 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14972603-914b-469f-b3eb-78f29f727bcf None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:27.984703 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14972603-914b-469f-b3eb-78f29f727bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:27.984990 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14972603-914b-469f-b3eb-78f29f727bcf None admin] PATH_INFO: `/v3/projects/82a0854bb4bb489680109c44becf6ac6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:27.986641 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14972603-914b-469f-b3eb-78f29f727bcf None admin] RBAC: Authorizing `identity:get_project(project_id=82a0854bb4bb489680109c44becf6ac6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:27.989054 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14972603-914b-469f-b3eb-78f29f727bcf None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:27.991311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 70/135] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:50:27 2022] GET /identity/v3/projects/82a0854bb4bb489680109c44becf6ac6 => generated 312 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:27.998735 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14972603-914b-469f-b3eb-78f29f727bcf None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:28.018202 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 66/136] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 15:50:27 2022] PUT /identity/v3/projects/8edda125d8bb41498eca65801bd10882/users/7c7ec8ec280649a8bf1f51745c94153b/roles/85dcd5509fca4911881d45baf5f1d5a5 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 15:50:28.053462 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:28.162909 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14972603-914b-469f-b3eb-78f29f727bcf None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:28.234206 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85429a3f-62e7-4ce1-bbe8-df9f74097fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:28.235210 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85429a3f-62e7-4ce1-bbe8-df9f74097fb7 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:28.235504 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85429a3f-62e7-4ce1-bbe8-df9f74097fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:28.235787 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85429a3f-62e7-4ce1-bbe8-df9f74097fb7 None admin] PATH_INFO: `/v3/projects/82a0854bb4bb489680109c44becf6ac6/users/7c7ec8ec280649a8bf1f51745c94153b/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:28.239314 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85429a3f-62e7-4ce1-bbe8-df9f74097fb7 None admin] RBAC: Authorizing `identity:create_grant(project_id=82a0854bb4bb489680109c44becf6ac6, user_id=7c7ec8ec280649a8bf1f51745c94153b, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:28.241760 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85429a3f-62e7-4ce1-bbe8-df9f74097fb7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:28.265681 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 71/137] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 15:50:27 2022] PUT /identity/v3/projects/82a0854bb4bb489680109c44becf6ac6/users/7c7ec8ec280649a8bf1f51745c94153b/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 15:50:28.453819 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d8e33-568b-45c4-94b8-c5e8ab4fff2c None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:28.454023 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d8e33-568b-45c4-94b8-c5e8ab4fff2c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:28.454376 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d8e33-568b-45c4-94b8-c5e8ab4fff2c None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:28.455671 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 67/138] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:28.462388 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e9921-f8a0-48c7-b9f7-f78aa1ed6e35 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:28.462940 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e9921-f8a0-48c7-b9f7-f78aa1ed6e35 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:28.462940 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e9921-f8a0-48c7-b9f7-f78aa1ed6e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:28.490830 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f74e9921-f8a0-48c7-b9f7-f78aa1ed6e35 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:28.492083 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f74e9921-f8a0-48c7-b9f7-f78aa1ed6e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:28.594001 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 72/139] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:28.619633 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d8e33-568b-45c4-94b8-c5e8ab4fff2c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:28.788973 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d8e33-568b-45c4-94b8-c5e8ab4fff2c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:28.840907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:28.859490 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8cc22b-18f7-4094-9370-eb3c31dc4541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:28.860307 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8cc22b-18f7-4094-9370-eb3c31dc4541 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:28.860596 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8cc22b-18f7-4094-9370-eb3c31dc4541 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:28.860876 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8cc22b-18f7-4094-9370-eb3c31dc4541 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:28.862157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8cc22b-18f7-4094-9370-eb3c31dc4541 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:28.864355 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8cc22b-18f7-4094-9370-eb3c31dc4541 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:28.866283 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 68/140] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:50:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:28.872087 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e9921-f8a0-48c7-b9f7-f78aa1ed6e35 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:28.910999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:29.013153 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e9921-f8a0-48c7-b9f7-f78aa1ed6e35 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:29.057244 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5622b4f-9e4a-44de-ae07-6c317bf06514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:29.057891 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5622b4f-9e4a-44de-ae07-6c317bf06514 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:29.058181 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5622b4f-9e4a-44de-ae07-6c317bf06514 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:29.058479 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5622b4f-9e4a-44de-ae07-6c317bf06514 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:29.059354 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5622b4f-9e4a-44de-ae07-6c317bf06514 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:29.060828 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5622b4f-9e4a-44de-ae07-6c317bf06514 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:29.090368 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 73/141] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 15:50:28 2022] POST /identity/v3/users => generated 283 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:50:29.465873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9d2df-9f28-42f0-89f5-8311483f7f64 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:29.466562 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9d2df-9f28-42f0-89f5-8311483f7f64 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:29.467048 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9d2df-9f28-42f0-89f5-8311483f7f64 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:29.468972 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 69/142] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:29.474451 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ede0d-4b62-48f8-a0bd-93cc36389e02 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:29.475096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ede0d-4b62-48f8-a0bd-93cc36389e02 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:29.475096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ede0d-4b62-48f8-a0bd-93cc36389e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:29.494970 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c68832-5b35-4b01-8671-008f443ba95a None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:29.495271 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c68832-5b35-4b01-8671-008f443ba95a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:29.495662 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c68832-5b35-4b01-8671-008f443ba95a None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:29.497166 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 70/143] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 15:50:29.502761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9e3ede0d-4b62-48f8-a0bd-93cc36389e02 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:29.504550 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9e3ede0d-4b62-48f8-a0bd-93cc36389e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:29.504964 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c54b2-20d2-4c4a-8d6e-b12a1c7c6258 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:29.505479 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c54b2-20d2-4c4a-8d6e-b12a1c7c6258 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:29.505959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c54b2-20d2-4c4a-8d6e-b12a1c7c6258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:29.531407 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a60c54b2-20d2-4c4a-8d6e-b12a1c7c6258 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:29.533121 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a60c54b2-20d2-4c4a-8d6e-b12a1c7c6258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:29.604963 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 74/144] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:29.643224 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3ede0d-4b62-48f8-a0bd-93cc36389e02 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:29.651942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 71/145] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:29.653945 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:29.682583 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c54b2-20d2-4c4a-8d6e-b12a1c7c6258 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:29.808831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3ede0d-4b62-48f8-a0bd-93cc36389e02 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:29.865293 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43941ea-8bf8-4500-9532-9d25ff65ba25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:29.866546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43941ea-8bf8-4500-9532-9d25ff65ba25 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:29.867025 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43941ea-8bf8-4500-9532-9d25ff65ba25 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:29.867587 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43941ea-8bf8-4500-9532-9d25ff65ba25 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:29.883101 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43941ea-8bf8-4500-9532-9d25ff65ba25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:29.892106 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b43941ea-8bf8-4500-9532-9d25ff65ba25 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:50:29.892106 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:29.892106 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:29.892106 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:29.892106 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:29.892106 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:29.892106 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:29.892106 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:29.892106 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:29.892106 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:29.892106 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:29.892106 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:29.892106 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:29.892106 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:29.892106 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:29.892106 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:29.892990 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:29.892990 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:29.892990 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:29.892990 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:29.892990 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:29.892990 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:29.892990 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:29.892990 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:29.892990 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:29.892990 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:29.892990 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:29.892990 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:29.892990 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:29.892990 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:29.892990 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:29.892990 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:29.894553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:29.894553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:29.894553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:29.894553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:29.894553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:29.894553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:29.894553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:29.894553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:29.894553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:29.894553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:29.894553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:50:29.894553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:50:29.897537 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 75/146] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:50:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:29.907181 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43941ea-8bf8-4500-9532-9d25ff65ba25 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:29.946634 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c54b2-20d2-4c4a-8d6e-b12a1c7c6258 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:29.998421 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:30.017395 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d8bdc-8a1c-42f5-af93-410945a3e364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:30.018617 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d8bdc-8a1c-42f5-af93-410945a3e364 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:30.019146 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d8bdc-8a1c-42f5-af93-410945a3e364 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:30.019629 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d8bdc-8a1c-42f5-af93-410945a3e364 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:30.044003 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6d8bdc-8a1c-42f5-af93-410945a3e364 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:30.052163 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:30.055980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-cc6d8bdc-8a1c-42f5-af93-410945a3e364 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:50:30.055980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:30.055980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:30.055980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:30.055980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:30.055980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:30.055980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:30.055980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:30.055980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:30.055980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:30.055980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:30.055980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:30.055980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:30.055980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:30.055980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:30.055980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:30.056907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:30.056907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:30.056907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:30.056907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:30.056907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:30.056907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:30.056907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:30.056907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:30.056907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:30.056907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:30.056907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:30.056907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:30.056907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:30.056907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:30.056907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:30.056907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:30.057882 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:30.057882 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:30.057882 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:30.057882 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:30.057882 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:30.057882 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:30.057882 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:30.057882 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:30.057882 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:30.057882 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:30.057882 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:50:30.057882 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:50:30.065753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 72/147] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:50:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:50:30.065753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d8bdc-8a1c-42f5-af93-410945a3e364 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:30.128328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43941ea-8bf8-4500-9532-9d25ff65ba25 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:30.184170 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e5a873-0232-49ac-b877-6a2dd25c0731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:30.185262 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5a873-0232-49ac-b877-6a2dd25c0731 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:30.185774 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5a873-0232-49ac-b877-6a2dd25c0731 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:30.186265 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5a873-0232-49ac-b877-6a2dd25c0731 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:30.187725 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e5a873-0232-49ac-b877-6a2dd25c0731 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:30.188256 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e5a873-0232-49ac-b877-6a2dd25c0731 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:30.189987 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e5a873-0232-49ac-b877-6a2dd25c0731 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:30.202441 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 76/148] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:50:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:30.208849 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e5a873-0232-49ac-b877-6a2dd25c0731 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:30.260299 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d8bdc-8a1c-42f5-af93-410945a3e364 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:30.327339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f4f5b-9a62-442f-9901-cc9e86eaa05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:30.327952 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f4f5b-9a62-442f-9901-cc9e86eaa05d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:30.328248 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f4f5b-9a62-442f-9901-cc9e86eaa05d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:30.328529 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f4f5b-9a62-442f-9901-cc9e86eaa05d None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:30.329305 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f4f5b-9a62-442f-9901-cc9e86eaa05d None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:30.329654 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f4f5b-9a62-442f-9901-cc9e86eaa05d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:30.331142 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f4f5b-9a62-442f-9901-cc9e86eaa05d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:30.351137 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 73/149] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 15:50:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:30.375123 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e5a873-0232-49ac-b877-6a2dd25c0731 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:30.385626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f4f5b-9a62-442f-9901-cc9e86eaa05d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:30.437317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:30.444633 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff54e7-d8cd-44f7-87a9-70d6c2cc569c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:30.445873 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff54e7-d8cd-44f7-87a9-70d6c2cc569c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:30.446353 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff54e7-d8cd-44f7-87a9-70d6c2cc569c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:30.446816 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff54e7-d8cd-44f7-87a9-70d6c2cc569c None admin] PATH_INFO: `/v3/users/bf570352584f4974b4b425f3e760bf0b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:30.448325 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff54e7-d8cd-44f7-87a9-70d6c2cc569c None admin] RBAC: Authorizing `identity:get_user(user_id=bf570352584f4974b4b425f3e760bf0b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:30.450039 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff54e7-d8cd-44f7-87a9-70d6c2cc569c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:30.451918 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 77/150] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:30 2022] GET /identity/v3/users/bf570352584f4974b4b425f3e760bf0b => generated 247 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:50:30.458213 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:30.458401 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff54e7-d8cd-44f7-87a9-70d6c2cc569c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:30.546350 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f4f5b-9a62-442f-9901-cc9e86eaa05d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:30.606784 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6604740f-45b8-44e7-a1fb-fd721b2e81d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:30.607638 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604740f-45b8-44e7-a1fb-fd721b2e81d3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:30.607926 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604740f-45b8-44e7-a1fb-fd721b2e81d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:30.608185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604740f-45b8-44e7-a1fb-fd721b2e81d3 None admin] PATH_INFO: `/v3/users/7c7ec8ec280649a8bf1f51745c94153b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:30.609613 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6604740f-45b8-44e7-a1fb-fd721b2e81d3 None admin] RBAC: Authorizing `identity:get_user(user_id=7c7ec8ec280649a8bf1f51745c94153b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:30.616623 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6604740f-45b8-44e7-a1fb-fd721b2e81d3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:30.616623 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 74/151] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:30 2022] GET /identity/v3/users/7c7ec8ec280649a8bf1f51745c94153b => generated 275 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:30.622112 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6604740f-45b8-44e7-a1fb-fd721b2e81d3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:30.663865 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff54e7-d8cd-44f7-87a9-70d6c2cc569c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:30.710665 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e9253b-1a2c-470a-b003-6abf700829b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:30.711703 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9253b-1a2c-470a-b003-6abf700829b3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:30.712208 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9253b-1a2c-470a-b003-6abf700829b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:30.712672 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9253b-1a2c-470a-b003-6abf700829b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:30.776617 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6604740f-45b8-44e7-a1fb-fd721b2e81d3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:30.792244 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:30.801611 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:30.804310 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e9253b-1a2c-470a-b003-6abf700829b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:30.806150 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e9253b-1a2c-470a-b003-6abf700829b3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:30.838351 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4d1b6-4c11-4ec7-b370-387ba064f4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:30.839031 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4d1b6-4c11-4ec7-b370-387ba064f4dd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:30.839330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4d1b6-4c11-4ec7-b370-387ba064f4dd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:30.839992 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4d1b6-4c11-4ec7-b370-387ba064f4dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:30.946421 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f4d1b6-4c11-4ec7-b370-387ba064f4dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:30.947856 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f4d1b6-4c11-4ec7-b370-387ba064f4dd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:30.964894 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 78/152] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:50:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 15:50:30.970998 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e9253b-1a2c-470a-b003-6abf700829b3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:31.094491 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 75/153] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:50:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 15:50:31.095884 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60017a2-5633-499b-ae8d-b41cfb9413a5 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:31.096173 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60017a2-5633-499b-ae8d-b41cfb9413a5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:31.096453 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60017a2-5633-499b-ae8d-b41cfb9413a5 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:31.098870 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 76/154] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:31.106131 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2ddc7-ac2b-4453-8782-6a778566e215 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:31.106421 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2ddc7-ac2b-4453-8782-6a778566e215 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:31.106709 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2ddc7-ac2b-4453-8782-6a778566e215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:31.140294 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e9253b-1a2c-470a-b003-6abf700829b3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:31.142733 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a7d2ddc7-ac2b-4453-8782-6a778566e215 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:31.144540 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a7d2ddc7-ac2b-4453-8782-6a778566e215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:31.146729 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:31.147876 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:31.251338 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc852dd8-0bd2-4640-a338-8c02841b078f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:31.252296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc852dd8-0bd2-4640-a338-8c02841b078f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:31.252586 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc852dd8-0bd2-4640-a338-8c02841b078f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:31.252864 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc852dd8-0bd2-4640-a338-8c02841b078f None admin] PATH_INFO: `/v3/projects/8edda125d8bb41498eca65801bd10882` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:31.258411 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc852dd8-0bd2-4640-a338-8c02841b078f None admin] RBAC: Authorizing `identity:get_project(project_id=8edda125d8bb41498eca65801bd10882)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:31.260102 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc852dd8-0bd2-4640-a338-8c02841b078f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:31.261876 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 77/155] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 21 15:50:31.262671 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 79/156] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:50:30 2022] GET /identity/v3/projects/8edda125d8bb41498eca65801bd10882 => generated 298 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:50:31.262774 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2ddc7-ac2b-4453-8782-6a778566e215 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:31.270677 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc852dd8-0bd2-4640-a338-8c02841b078f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:31.476578 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2ddc7-ac2b-4453-8782-6a778566e215 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:31.501524 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc852dd8-0bd2-4640-a338-8c02841b078f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:31.547462 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18537814-7ff7-4f2b-9e6a-5547d8f63de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:31.548352 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18537814-7ff7-4f2b-9e6a-5547d8f63de3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:31.548640 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18537814-7ff7-4f2b-9e6a-5547d8f63de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:31.548917 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18537814-7ff7-4f2b-9e6a-5547d8f63de3 None admin] PATH_INFO: `/v3/projects/8edda125d8bb41498eca65801bd10882` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:31.557603 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18537814-7ff7-4f2b-9e6a-5547d8f63de3 None admin] RBAC: Authorizing `identity:get_project(project_id=8edda125d8bb41498eca65801bd10882)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:31.557603 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18537814-7ff7-4f2b-9e6a-5547d8f63de3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:31.557887 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 78/157] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:50:31 2022] GET /identity/v3/projects/8edda125d8bb41498eca65801bd10882 => generated 298 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:31.558595 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18537814-7ff7-4f2b-9e6a-5547d8f63de3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:31.578566 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:31.585754 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd46c2-dd05-49c6-aad6-684f3cc97d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:31.586761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd46c2-dd05-49c6-aad6-684f3cc97d5f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:31.587260 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd46c2-dd05-49c6-aad6-684f3cc97d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:31.587726 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd46c2-dd05-49c6-aad6-684f3cc97d5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:31.588768 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bd46c2-dd05-49c6-aad6-684f3cc97d5f None admin] RBAC: Adding query filter params (role.id=475521b8a0174037b162f92022aea1a1, scope.project.id=8edda125d8bb41498eca65801bd10882, user.id=bf570352584f4974b4b425f3e760bf0b) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:31.589276 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bd46c2-dd05-49c6-aad6-684f3cc97d5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:31.595160 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bd46c2-dd05-49c6-aad6-684f3cc97d5f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:31.604702 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 80/158] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:50:31 2022] GET /identity/v3/role_assignments?user.id=bf570352584f4974b4b425f3e760bf0b&scope.project.id=8edda125d8bb41498eca65801bd10882&role.id=475521b8a0174037b162f92022aea1a1 => generated 608 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:31.606000 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd46c2-dd05-49c6-aad6-684f3cc97d5f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:31.702168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:31.737586 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18537814-7ff7-4f2b-9e6a-5547d8f63de3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:31.792165 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb8a8b-fe2f-4866-b0bc-a248fe31a495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:31.792979 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb8a8b-fe2f-4866-b0bc-a248fe31a495 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:31.793173 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb8a8b-fe2f-4866-b0bc-a248fe31a495 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:31.793449 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb8a8b-fe2f-4866-b0bc-a248fe31a495 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:31.815674 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb8a8b-fe2f-4866-b0bc-a248fe31a495 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:31.827946 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-0ecb8a8b-fe2f-4866-b0bc-a248fe31a495 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 15:50:31.827946 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:31.827946 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:31.827946 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:31.827946 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:31.827946 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:31.827946 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:31.827946 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:31.827946 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:31.827946 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:31.827946 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:31.827946 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:31.827946 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:31.827946 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:31.827946 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:31.827946 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:31.829651 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:31.829651 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:31.829651 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:31.829651 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:31.829651 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:31.829651 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:31.829651 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:31.829651 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:31.829651 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:31.829651 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:31.829651 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:31.829651 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:31.829651 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:31.829651 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:31.829651 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:31.829651 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:31.831087 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:31.831087 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:31.831087 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:31.831087 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:31.831087 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:31.831087 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:31.831087 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:31.831087 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:31.831087 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:31.831087 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:31.831087 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 15:50:31.831087 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:50:31.831087 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 79/159] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:50:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:31.847135 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb8a8b-fe2f-4866-b0bc-a248fe31a495 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:31.935853 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd46c2-dd05-49c6-aad6-684f3cc97d5f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:31.953618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:32.001745 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb8a8b-fe2f-4866-b0bc-a248fe31a495 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:32.041588 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d29e200-6de9-4eae-ba0b-ca934344e2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:32.043015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29e200-6de9-4eae-ba0b-ca934344e2e2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:32.043744 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29e200-6de9-4eae-ba0b-ca934344e2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:32.044030 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29e200-6de9-4eae-ba0b-ca934344e2e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:32.046403 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d29e200-6de9-4eae-ba0b-ca934344e2e2 None admin] RBAC: Adding query filter params (role.id=54195e660fc349978b39d5794765a095, scope.project.id=8edda125d8bb41498eca65801bd10882, user.id=7c7ec8ec280649a8bf1f51745c94153b) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:32.046742 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d29e200-6de9-4eae-ba0b-ca934344e2e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:32.048236 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d29e200-6de9-4eae-ba0b-ca934344e2e2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:32.050325 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b667cf5-cd92-4d7c-b195-2a03a9406468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:32.051049 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b667cf5-cd92-4d7c-b195-2a03a9406468 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:32.051340 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b667cf5-cd92-4d7c-b195-2a03a9406468 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:32.051624 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b667cf5-cd92-4d7c-b195-2a03a9406468 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:32.052493 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b667cf5-cd92-4d7c-b195-2a03a9406468 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:32.052810 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b667cf5-cd92-4d7c-b195-2a03a9406468 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:32.054295 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b667cf5-cd92-4d7c-b195-2a03a9406468 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:32.062198 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 81/160] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:50:31 2022] GET /identity/v3/role_assignments?user.id=7c7ec8ec280649a8bf1f51745c94153b&scope.project.id=8edda125d8bb41498eca65801bd10882&role.id=54195e660fc349978b39d5794765a095 => generated 608 bytes in 457 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:32.063551 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafdeec2-3fbd-49dd-ba2f-ef46296b90be None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:32.063841 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafdeec2-3fbd-49dd-ba2f-ef46296b90be None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:32.064125 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafdeec2-3fbd-49dd-ba2f-ef46296b90be None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:32.073695 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 82/161] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:32 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:32.080594 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 80/162] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 15:50:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:50:32.081298 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7738748-9640-48e2-baeb-f5c05ff8af5f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:32.085964 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7738748-9640-48e2-baeb-f5c05ff8af5f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:32.086350 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7738748-9640-48e2-baeb-f5c05ff8af5f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:32.088117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 83/163] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:32 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:32.088962 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b667cf5-cd92-4d7c-b195-2a03a9406468 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:32.093843 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171fb549-3918-4d00-b14d-68521c2caf79 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:32.094141 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171fb549-3918-4d00-b14d-68521c2caf79 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:32.094417 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171fb549-3918-4d00-b14d-68521c2caf79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:32.136227 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-171fb549-3918-4d00-b14d-68521c2caf79 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:32.137510 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-171fb549-3918-4d00-b14d-68521c2caf79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:32.289472 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b667cf5-cd92-4d7c-b195-2a03a9406468 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:32.329152 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:32.342110 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34636d-3deb-4b91-b295-cd2dda6b5596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:32.343291 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34636d-3deb-4b91-b295-cd2dda6b5596 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:32.343655 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34636d-3deb-4b91-b295-cd2dda6b5596 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:32.343942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34636d-3deb-4b91-b295-cd2dda6b5596 None admin] PATH_INFO: `/v3/users/3418daf276a84936bf5896c885e85a1f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:32.345464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee34636d-3deb-4b91-b295-cd2dda6b5596 None admin] RBAC: Authorizing `identity:get_user(user_id=3418daf276a84936bf5896c885e85a1f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:32.345805 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 84/164] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:32.347475 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee34636d-3deb-4b91-b295-cd2dda6b5596 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:32.348117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5632a8-097f-4247-84d4-23665a5bcee6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:32.348856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5632a8-097f-4247-84d4-23665a5bcee6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:32.349325 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5632a8-097f-4247-84d4-23665a5bcee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:32.350584 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 81/165] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:32 2022] GET /identity/v3/users/3418daf276a84936bf5896c885e85a1f => generated 289 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:32.355460 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34636d-3deb-4b91-b295-cd2dda6b5596 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:32.378877 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ff5632a8-097f-4247-84d4-23665a5bcee6 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:32.380454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ff5632a8-097f-4247-84d4-23665a5bcee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:32.406174 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:32.479023 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 85/166] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:32.483438 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5632a8-097f-4247-84d4-23665a5bcee6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:32.494644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34636d-3deb-4b91-b295-cd2dda6b5596 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:32.541084 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ca01c-7d05-45b8-9a01-33f4c0963fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:32.542002 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ca01c-7d05-45b8-9a01-33f4c0963fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:32.542296 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ca01c-7d05-45b8-9a01-33f4c0963fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:32.542631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ca01c-7d05-45b8-9a01-33f4c0963fe6 None admin] PATH_INFO: `/v3/projects/8edda125d8bb41498eca65801bd10882` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:32.544044 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ca01c-7d05-45b8-9a01-33f4c0963fe6 None admin] RBAC: Authorizing `identity:get_project(project_id=8edda125d8bb41498eca65801bd10882)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:32.545826 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ca01c-7d05-45b8-9a01-33f4c0963fe6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:32.547588 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 82/167] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:50:32 2022] GET /identity/v3/projects/8edda125d8bb41498eca65801bd10882 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:50:32.554537 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ca01c-7d05-45b8-9a01-33f4c0963fe6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:32.613805 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5632a8-097f-4247-84d4-23665a5bcee6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:32.640782 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:32.651691 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80cb06f-fd9e-4b01-9e93-c978b855803c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:32.652863 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80cb06f-fd9e-4b01-9e93-c978b855803c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:32.653340 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80cb06f-fd9e-4b01-9e93-c978b855803c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:32.653977 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80cb06f-fd9e-4b01-9e93-c978b855803c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:32.666406 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80cb06f-fd9e-4b01-9e93-c978b855803c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:32.674602 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-f80cb06f-fd9e-4b01-9e93-c978b855803c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:50:32.674602 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:32.674602 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:32.674602 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:32.674602 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:32.674602 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:32.674602 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:32.674602 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:32.674602 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:32.674602 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:32.674602 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:32.674602 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:32.674602 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:32.674602 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:32.674602 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:32.674602 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:32.676248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:32.676248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:32.676248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:32.676248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:32.676248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:32.676248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:32.676248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:32.676248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:32.676248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:32.676248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:32.676248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:32.676248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:32.676248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:32.676248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:32.676248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:32.676248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:32.677067 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:32.677067 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:32.677067 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:32.677067 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:32.677067 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:32.677067 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:32.677067 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:32.677067 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:32.677067 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:32.677067 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:32.677067 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:50:32.677067 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:50:32.679648 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 86/168] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:50:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:32.680791 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80cb06f-fd9e-4b01-9e93-c978b855803c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:32.686820 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ca01c-7d05-45b8-9a01-33f4c0963fe6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:32.699124 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:32.757323 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc734c-4e15-4f53-8edd-bfbc863a1d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:32.758401 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc734c-4e15-4f53-8edd-bfbc863a1d1a None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:32.758690 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc734c-4e15-4f53-8edd-bfbc863a1d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:32.758970 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc734c-4e15-4f53-8edd-bfbc863a1d1a None admin] PATH_INFO: `/v3/projects/8edda125d8bb41498eca65801bd10882/users/3418daf276a84936bf5896c885e85a1f/roles/f4d0b371e4b140d9a9f6d9c6983c0ec8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:32.766729 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc734c-4e15-4f53-8edd-bfbc863a1d1a None admin] RBAC: Authorizing `identity:create_grant(project_id=8edda125d8bb41498eca65801bd10882, user_id=3418daf276a84936bf5896c885e85a1f, role_id=f4d0b371e4b140d9a9f6d9c6983c0ec8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:32.768369 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc734c-4e15-4f53-8edd-bfbc863a1d1a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:32.793501 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80cb06f-fd9e-4b01-9e93-c978b855803c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:32.799374 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 83/169] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 15:50:32 2022] PUT /identity/v3/projects/8edda125d8bb41498eca65801bd10882/users/3418daf276a84936bf5896c885e85a1f/roles/f4d0b371e4b140d9a9f6d9c6983c0ec8 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 15:50:32.800929 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc734c-4e15-4f53-8edd-bfbc863a1d1a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:32.843873 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73e107-ae77-4d5c-9937-69c8d25eeec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:32.845100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73e107-ae77-4d5c-9937-69c8d25eeec8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:32.847438 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73e107-ae77-4d5c-9937-69c8d25eeec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:32.847995 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73e107-ae77-4d5c-9937-69c8d25eeec8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:32.878797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a73e107-ae77-4d5c-9937-69c8d25eeec8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:32.905156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-0a73e107-ae77-4d5c-9937-69c8d25eeec8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 15:50:32.905156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:32.905156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:32.905156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:32.905156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:32.905156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:32.905156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:32.905156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:32.905156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:32.905156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:32.905156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:32.905156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:32.905156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:32.905156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:32.905156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:32.905156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:32.906319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:32.906319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:32.906319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:32.906319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:32.906319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:32.906319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:32.906319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:32.906319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:32.906319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:32.906319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:32.906319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:32.906319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:32.906319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:32.906319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:32.906319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:32.906319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:32.907145 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:32.907145 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:32.907145 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:32.907145 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:32.907145 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:32.907145 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:32.907145 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:32.907145 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:32.907145 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:32.907145 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:32.907145 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 15:50:32.907145 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:50:32.912459 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 87/170] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 15:50:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:32.920304 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73e107-ae77-4d5c-9937-69c8d25eeec8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:33.048734 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc734c-4e15-4f53-8edd-bfbc863a1d1a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:33.070519 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:33.107311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73e107-ae77-4d5c-9937-69c8d25eeec8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:33.135501 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:33.168176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1719d9-30dd-4ad8-aaad-686cfd00fc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:33.168885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1719d9-30dd-4ad8-aaad-686cfd00fc7e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:33.169169 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1719d9-30dd-4ad8-aaad-686cfd00fc7e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:33.169445 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1719d9-30dd-4ad8-aaad-686cfd00fc7e None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:33.170498 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1719d9-30dd-4ad8-aaad-686cfd00fc7e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:33.171006 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1719d9-30dd-4ad8-aaad-686cfd00fc7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:33.173448 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1719d9-30dd-4ad8-aaad-686cfd00fc7e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:33.179963 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94489e-950d-4704-83d3-efcf2f89760c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:33.181083 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94489e-950d-4704-83d3-efcf2f89760c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:33.181504 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94489e-950d-4704-83d3-efcf2f89760c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:33.182009 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94489e-950d-4704-83d3-efcf2f89760c None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:33.183280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc94489e-950d-4704-83d3-efcf2f89760c None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:33.183768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc94489e-950d-4704-83d3-efcf2f89760c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:33.186227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc94489e-950d-4704-83d3-efcf2f89760c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:33.202515 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 88/171] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:50:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:33.205015 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 84/172] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 15:50:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:33.208933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1719d9-30dd-4ad8-aaad-686cfd00fc7e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:33.212782 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94489e-950d-4704-83d3-efcf2f89760c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:33.402433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1719d9-30dd-4ad8-aaad-686cfd00fc7e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:33.466396 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c63cb4-1b06-4b31-868e-a3d0a3a2184b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:33.467719 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c63cb4-1b06-4b31-868e-a3d0a3a2184b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:33.468210 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c63cb4-1b06-4b31-868e-a3d0a3a2184b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:33.468690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c63cb4-1b06-4b31-868e-a3d0a3a2184b None admin] PATH_INFO: `/v3/users/7c7ec8ec280649a8bf1f51745c94153b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:33.470493 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c63cb4-1b06-4b31-868e-a3d0a3a2184b None admin] RBAC: Authorizing `identity:get_user(user_id=7c7ec8ec280649a8bf1f51745c94153b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:33.472354 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c63cb4-1b06-4b31-868e-a3d0a3a2184b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:33.474539 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 89/173] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:33 2022] GET /identity/v3/users/7c7ec8ec280649a8bf1f51745c94153b => generated 275 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:33.480061 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c63cb4-1b06-4b31-868e-a3d0a3a2184b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:33.485795 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:33.496980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94489e-950d-4704-83d3-efcf2f89760c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:33.536826 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2876c8-154b-4ce4-8b22-e04b6c0a070e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:33.537859 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2876c8-154b-4ce4-8b22-e04b6c0a070e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:33.538143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2876c8-154b-4ce4-8b22-e04b6c0a070e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:33.538422 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2876c8-154b-4ce4-8b22-e04b6c0a070e None admin] PATH_INFO: `/v3/users/7c7ec8ec280649a8bf1f51745c94153b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:33.539699 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2876c8-154b-4ce4-8b22-e04b6c0a070e None admin] RBAC: Authorizing `identity:get_user(user_id=7c7ec8ec280649a8bf1f51745c94153b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:33.545167 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2876c8-154b-4ce4-8b22-e04b6c0a070e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:33.545167 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 85/174] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:33 2022] GET /identity/v3/users/7c7ec8ec280649a8bf1f51745c94153b => generated 275 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:33.548692 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2876c8-154b-4ce4-8b22-e04b6c0a070e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:33.610355 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:33.657119 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2876c8-154b-4ce4-8b22-e04b6c0a070e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:33.660135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c63cb4-1b06-4b31-868e-a3d0a3a2184b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:33.701376 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44702b0-b497-47a9-bf2c-fdc0f5eced45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:33.702124 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34960b1f-3a54-47ec-a2d2-95bf8f3d0298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:33.703196 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34960b1f-3a54-47ec-a2d2-95bf8f3d0298 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:33.703588 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44702b0-b497-47a9-bf2c-fdc0f5eced45 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:33.703715 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34960b1f-3a54-47ec-a2d2-95bf8f3d0298 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:33.704034 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34960b1f-3a54-47ec-a2d2-95bf8f3d0298 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:33.704705 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44702b0-b497-47a9-bf2c-fdc0f5eced45 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:33.705452 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44702b0-b497-47a9-bf2c-fdc0f5eced45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:33.785360 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34960b1f-3a54-47ec-a2d2-95bf8f3d0298 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:33.786880 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34960b1f-3a54-47ec-a2d2-95bf8f3d0298 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:33.791459 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44702b0-b497-47a9-bf2c-fdc0f5eced45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:33.793246 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44702b0-b497-47a9-bf2c-fdc0f5eced45 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:33.796578 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:33.854764 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:33.902405 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 86/175] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:50:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 15:50:33.908494 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bbeeff-74cb-462d-af0e-556bd49ba8a9 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:33.908791 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bbeeff-74cb-462d-af0e-556bd49ba8a9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:33.908985 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bbeeff-74cb-462d-af0e-556bd49ba8a9 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:33.910516 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 87/176] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:33.912008 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55970e1d-b716-478a-9391-d25d9a6d9489 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:33.912295 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55970e1d-b716-478a-9391-d25d9a6d9489 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:33.912679 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55970e1d-b716-478a-9391-d25d9a6d9489 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:33.914037 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 88/177] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:33.918580 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55970e1d-b716-478a-9391-d25d9a6d9489 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:33.936712 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 90/178] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:50:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 15:50:33.938577 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14898b8-fb71-4d5c-8ce8-aa84a338a5f7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:33.939180 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14898b8-fb71-4d5c-8ce8-aa84a338a5f7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:33.939683 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14898b8-fb71-4d5c-8ce8-aa84a338a5f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:33.959001 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a14898b8-fb71-4d5c-8ce8-aa84a338a5f7 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:33.960546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a14898b8-fb71-4d5c-8ce8-aa84a338a5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:34.033773 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55970e1d-b716-478a-9391-d25d9a6d9489 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:34.040259 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 91/179] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:34.042104 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434636ad-9c84-487c-a97a-c5b60267be2a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:34.042644 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434636ad-9c84-487c-a97a-c5b60267be2a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:34.043127 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434636ad-9c84-487c-a97a-c5b60267be2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:34.063015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-434636ad-9c84-487c-a97a-c5b60267be2a None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:34.064646 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-434636ad-9c84-487c-a97a-c5b60267be2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:34.071962 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0bc88d-ea15-4d02-9948-fe138f520902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:34.072892 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bc88d-ea15-4d02-9948-fe138f520902 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:34.073180 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bc88d-ea15-4d02-9948-fe138f520902 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:34.073457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bc88d-ea15-4d02-9948-fe138f520902 None admin] PATH_INFO: `/v3/projects/82a0854bb4bb489680109c44becf6ac6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:34.074735 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0bc88d-ea15-4d02-9948-fe138f520902 None admin] RBAC: Authorizing `identity:get_project(project_id=82a0854bb4bb489680109c44becf6ac6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:34.076220 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0bc88d-ea15-4d02-9948-fe138f520902 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:34.078198 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 89/180] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:50:33 2022] GET /identity/v3/projects/82a0854bb4bb489680109c44becf6ac6 => generated 312 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:34.082247 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0bc88d-ea15-4d02-9948-fe138f520902 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:34.101798 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:34.117089 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:34.156176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 92/181] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:34.157417 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434636ad-9c84-487c-a97a-c5b60267be2a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:34.218845 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0bc88d-ea15-4d02-9948-fe138f520902 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:34.260438 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37980f1a-1eef-4f6e-b844-85a25320662b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:34.261466 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37980f1a-1eef-4f6e-b844-85a25320662b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:34.261854 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37980f1a-1eef-4f6e-b844-85a25320662b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:34.262261 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37980f1a-1eef-4f6e-b844-85a25320662b None admin] PATH_INFO: `/v3/projects/8edda125d8bb41498eca65801bd10882` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:34.264172 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37980f1a-1eef-4f6e-b844-85a25320662b None admin] RBAC: Authorizing `identity:get_project(project_id=8edda125d8bb41498eca65801bd10882)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:34.266718 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37980f1a-1eef-4f6e-b844-85a25320662b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:34.269831 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 90/182] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:50:34 2022] GET /identity/v3/projects/8edda125d8bb41498eca65801bd10882 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:34.271281 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37980f1a-1eef-4f6e-b844-85a25320662b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:34.276539 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434636ad-9c84-487c-a97a-c5b60267be2a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:34.316910 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25614fc-1dbe-4eed-8dcc-7f3af57155e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:34.318325 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25614fc-1dbe-4eed-8dcc-7f3af57155e7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:34.318817 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25614fc-1dbe-4eed-8dcc-7f3af57155e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:34.319467 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25614fc-1dbe-4eed-8dcc-7f3af57155e7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:34.332609 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25614fc-1dbe-4eed-8dcc-7f3af57155e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:34.341978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-a25614fc-1dbe-4eed-8dcc-7f3af57155e7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 15:50:34.341978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:34.341978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:34.341978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:34.341978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:34.341978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:34.341978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:34.341978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:34.341978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:34.341978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:34.341978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:34.341978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:34.341978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:34.341978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:34.341978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:34.341978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:34.342953 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:34.342953 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:34.342953 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:34.342953 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:34.342953 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:34.342953 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:34.342953 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:34.342953 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:34.342953 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:34.342953 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:34.342953 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:34.342953 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:34.342953 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:34.342953 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:34.342953 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:34.342953 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:34.343782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:34.343782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:34.343782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:34.343782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:34.343782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:34.343782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:34.343782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:34.343782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:34.343782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:34.343782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:34.343782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 15:50:34.343782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:50:34.346821 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 93/183] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 15:50:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:34.348112 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25614fc-1dbe-4eed-8dcc-7f3af57155e7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:34.371449 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:34.381402 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:34.381617 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37980f1a-1eef-4f6e-b844-85a25320662b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:34.417811 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60f3e8-ba64-4d27-a2c3-5fec9853ac88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:34.418583 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60f3e8-ba64-4d27-a2c3-5fec9853ac88 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:34.419001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60f3e8-ba64-4d27-a2c3-5fec9853ac88 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:34.419361 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60f3e8-ba64-4d27-a2c3-5fec9853ac88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:34.420223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee60f3e8-ba64-4d27-a2c3-5fec9853ac88 None admin] RBAC: Adding query filter params (role.id=54195e660fc349978b39d5794765a095, scope.project.id=82a0854bb4bb489680109c44becf6ac6, user.id=7c7ec8ec280649a8bf1f51745c94153b) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:34.420540 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee60f3e8-ba64-4d27-a2c3-5fec9853ac88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:34.422040 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee60f3e8-ba64-4d27-a2c3-5fec9853ac88 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:34.428576 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 91/184] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:50:34 2022] GET /identity/v3/role_assignments?user.id=7c7ec8ec280649a8bf1f51745c94153b&scope.project.id=82a0854bb4bb489680109c44becf6ac6&role.id=54195e660fc349978b39d5794765a095 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:34.434797 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60f3e8-ba64-4d27-a2c3-5fec9853ac88 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:34.500103 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25614fc-1dbe-4eed-8dcc-7f3af57155e7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:34.549700 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e481eba-27ba-4a5b-a38c-f38df683ec67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:34.551165 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e481eba-27ba-4a5b-a38c-f38df683ec67 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:34.551483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e481eba-27ba-4a5b-a38c-f38df683ec67 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:34.551881 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e481eba-27ba-4a5b-a38c-f38df683ec67 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:34.569956 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e481eba-27ba-4a5b-a38c-f38df683ec67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:34.580612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-0e481eba-27ba-4a5b-a38c-f38df683ec67 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:50:34.580612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:34.580612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:34.580612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:34.580612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:34.580612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:34.580612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:34.580612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:34.580612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:34.580612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:34.580612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:34.580612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:34.580612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:34.580612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:34.580612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:34.580612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:34.581699 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:34.581699 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:34.581699 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:34.581699 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:34.581699 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:34.581699 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:34.581699 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:34.581699 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:34.581699 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:34.581699 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:34.581699 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:34.581699 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:34.581699 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:34.581699 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:34.581699 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:34.581699 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:34.582513 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:34.582513 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:34.582513 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:34.582513 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:34.582513 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:34.582513 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:34.582513 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:34.582513 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:34.582513 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:34.582513 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:34.582513 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:50:34.582513 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:50:34.583135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 94/185] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:50:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:34.583963 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e481eba-27ba-4a5b-a38c-f38df683ec67 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:34.605279 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60f3e8-ba64-4d27-a2c3-5fec9853ac88 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:34.650087 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7cd00d-3305-4ed6-baf8-9da7f4928ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:34.650767 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cd00d-3305-4ed6-baf8-9da7f4928ddc None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:34.651058 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cd00d-3305-4ed6-baf8-9da7f4928ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:34.651340 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cd00d-3305-4ed6-baf8-9da7f4928ddc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:34.652209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7cd00d-3305-4ed6-baf8-9da7f4928ddc None admin] RBAC: Adding query filter params (role.id=85dcd5509fca4911881d45baf5f1d5a5, scope.project.id=8edda125d8bb41498eca65801bd10882, user.id=7c7ec8ec280649a8bf1f51745c94153b) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:34.652532 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7cd00d-3305-4ed6-baf8-9da7f4928ddc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:34.654047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7cd00d-3305-4ed6-baf8-9da7f4928ddc None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:34.663323 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 92/186] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:50:34 2022] GET /identity/v3/role_assignments?user.id=7c7ec8ec280649a8bf1f51745c94153b&scope.project.id=8edda125d8bb41498eca65801bd10882&role.id=85dcd5509fca4911881d45baf5f1d5a5 => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:34.664775 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f4d4b-4e1e-4210-a53d-2607f208d890 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:34.665075 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f4d4b-4e1e-4210-a53d-2607f208d890 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:34.665360 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f4d4b-4e1e-4210-a53d-2607f208d890 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:34.667043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 93/187] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:34.667856 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f4d4b-4e1e-4210-a53d-2607f208d890 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:34.700922 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:34.718769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:34.751787 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e481eba-27ba-4a5b-a38c-f38df683ec67 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:34.808897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1857f-4140-4eed-9b04-c15811c38f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:34.809698 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1857f-4140-4eed-9b04-c15811c38f9d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:34.809925 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1857f-4140-4eed-9b04-c15811c38f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:34.810306 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1857f-4140-4eed-9b04-c15811c38f9d None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:34.811217 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b1857f-4140-4eed-9b04-c15811c38f9d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:34.811536 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b1857f-4140-4eed-9b04-c15811c38f9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:34.812998 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b1857f-4140-4eed-9b04-c15811c38f9d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:34.815198 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f4d4b-4e1e-4210-a53d-2607f208d890 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:34.823603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 95/188] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:50:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:34.825119 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e2ad1-9012-48fc-ab97-4b48c71a24b6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:34.825415 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e2ad1-9012-48fc-ab97-4b48c71a24b6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:34.825723 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e2ad1-9012-48fc-ab97-4b48c71a24b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:34.851088 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-307e2ad1-9012-48fc-ab97-4b48c71a24b6 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:34.852750 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-307e2ad1-9012-48fc-ab97-4b48c71a24b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:34.859698 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1445c0-71b0-4449-8b93-3e9aaefdca2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:34.860439 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1445c0-71b0-4449-8b93-3e9aaefdca2f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:34.860774 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1445c0-71b0-4449-8b93-3e9aaefdca2f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:34.861061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1445c0-71b0-4449-8b93-3e9aaefdca2f None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:34.861866 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1445c0-71b0-4449-8b93-3e9aaefdca2f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:34.862239 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1445c0-71b0-4449-8b93-3e9aaefdca2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:34.863902 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1445c0-71b0-4449-8b93-3e9aaefdca2f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:34.877537 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 94/189] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:50:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:34.879288 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1445c0-71b0-4449-8b93-3e9aaefdca2f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:34.966733 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 96/190] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:34.968006 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e2ad1-9012-48fc-ab97-4b48c71a24b6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:35.026624 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:35.050429 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1445c0-71b0-4449-8b93-3e9aaefdca2f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:35.064192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:35.099904 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e2ad1-9012-48fc-ab97-4b48c71a24b6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:35.104108 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac09098c-8c0b-4f91-a5ba-fa2432b6785a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:35.105218 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac09098c-8c0b-4f91-a5ba-fa2432b6785a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:35.105501 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac09098c-8c0b-4f91-a5ba-fa2432b6785a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:35.105942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac09098c-8c0b-4f91-a5ba-fa2432b6785a None admin] PATH_INFO: `/v3/users/a75fbca2c9d442b2bca8554b27dc944a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:35.124315 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac09098c-8c0b-4f91-a5ba-fa2432b6785a None admin] RBAC: Authorizing `identity:get_user(user_id=a75fbca2c9d442b2bca8554b27dc944a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:35.126008 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac09098c-8c0b-4f91-a5ba-fa2432b6785a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:35.127787 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 95/191] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:34 2022] GET /identity/v3/users/a75fbca2c9d442b2bca8554b27dc944a => generated 283 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:35.129107 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac09098c-8c0b-4f91-a5ba-fa2432b6785a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:35.147796 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8f628-d7b9-41c9-b059-4bf486c685a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:35.148752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8f628-d7b9-41c9-b059-4bf486c685a6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:35.149060 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8f628-d7b9-41c9-b059-4bf486c685a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:35.149341 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8f628-d7b9-41c9-b059-4bf486c685a6 None admin] PATH_INFO: `/v3/users/a75fbca2c9d442b2bca8554b27dc944a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:35.150623 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8f628-d7b9-41c9-b059-4bf486c685a6 None admin] RBAC: Authorizing `identity:get_user(user_id=a75fbca2c9d442b2bca8554b27dc944a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:35.152095 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8f628-d7b9-41c9-b059-4bf486c685a6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:35.159991 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 97/192] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:34 2022] GET /identity/v3/users/a75fbca2c9d442b2bca8554b27dc944a => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:35.159991 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8f628-d7b9-41c9-b059-4bf486c685a6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:35.263477 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac09098c-8c0b-4f91-a5ba-fa2432b6785a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:35.284140 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8f628-d7b9-41c9-b059-4bf486c685a6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:35.309513 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d085f8-5d38-49ad-9d9b-3a67fd8a6c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:35.310386 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d085f8-5d38-49ad-9d9b-3a67fd8a6c60 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:35.310678 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d085f8-5d38-49ad-9d9b-3a67fd8a6c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:35.310959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d085f8-5d38-49ad-9d9b-3a67fd8a6c60 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:35.312166 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d085f8-5d38-49ad-9d9b-3a67fd8a6c60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:35.313664 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d085f8-5d38-49ad-9d9b-3a67fd8a6c60 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:35.315315 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 96/193] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:50:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:35.316644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d085f8-5d38-49ad-9d9b-3a67fd8a6c60 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:35.337300 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:35.346147 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df631500-cb20-4a1a-aa05-67745215d487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:35.346959 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df631500-cb20-4a1a-aa05-67745215d487 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:35.347361 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df631500-cb20-4a1a-aa05-67745215d487 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:35.347739 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df631500-cb20-4a1a-aa05-67745215d487 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:35.372914 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:35.479850 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df631500-cb20-4a1a-aa05-67745215d487 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:35.481957 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df631500-cb20-4a1a-aa05-67745215d487 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:35.485768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d085f8-5d38-49ad-9d9b-3a67fd8a6c60 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:35.540450 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f524a046-5c47-4d9a-b456-999de2d53326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:35.541484 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524a046-5c47-4d9a-b456-999de2d53326 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:35.542245 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524a046-5c47-4d9a-b456-999de2d53326 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:35.542633 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524a046-5c47-4d9a-b456-999de2d53326 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:35.634247 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f524a046-5c47-4d9a-b456-999de2d53326 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:35.634443 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 98/194] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:50:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 15:50:35.635094 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df631500-cb20-4a1a-aa05-67745215d487 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:35.635721 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f524a046-5c47-4d9a-b456-999de2d53326 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:35.646328 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:35.687403 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:35.743966 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 97/195] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:50:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 15:50:35.745232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df631500-cb20-4a1a-aa05-67745215d487 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:35.745802 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490df18-7b20-4f83-a377-6ea74c51c376 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:35.746103 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490df18-7b20-4f83-a377-6ea74c51c376 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:35.746449 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490df18-7b20-4f83-a377-6ea74c51c376 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:35.748156 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 98/196] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:35.748669 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3490df18-7b20-4f83-a377-6ea74c51c376 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:35.784232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868811cc-3095-4d89-810c-912a3f1c8fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:35.785395 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868811cc-3095-4d89-810c-912a3f1c8fa2 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:35.785710 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868811cc-3095-4d89-810c-912a3f1c8fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:35.785985 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868811cc-3095-4d89-810c-912a3f1c8fa2 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:35.786719 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868811cc-3095-4d89-810c-912a3f1c8fa2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:35.792234 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868811cc-3095-4d89-810c-912a3f1c8fa2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:35.815170 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 99/197] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 15:50:35 2022] POST /identity/v3/users => generated 297 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:50:35.817986 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868811cc-3095-4d89-810c-912a3f1c8fa2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:35.875777 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3490df18-7b20-4f83-a377-6ea74c51c376 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:35.890837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:35.925928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e215c-ae08-4d97-bc20-23e7a6603ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:35.926941 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e215c-ae08-4d97-bc20-23e7a6603ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:35.927226 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e215c-ae08-4d97-bc20-23e7a6603ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:35.927506 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e215c-ae08-4d97-bc20-23e7a6603ad9 None admin] PATH_INFO: `/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:35.929465 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019e215c-ae08-4d97-bc20-23e7a6603ad9 None admin] RBAC: Authorizing `identity:get_project(project_id=c9f5c8ab86d44f42af60e2d8442a4f3d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:35.931422 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019e215c-ae08-4d97-bc20-23e7a6603ad9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:35.933612 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 99/198] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:50:35 2022] GET /identity/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:35.934979 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669e2cb-8772-4dc7-a1b2-7aa1b8194ff5 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:35.935275 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669e2cb-8772-4dc7-a1b2-7aa1b8194ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:35.935562 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669e2cb-8772-4dc7-a1b2-7aa1b8194ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:35.951740 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:35.954697 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868811cc-3095-4d89-810c-912a3f1c8fa2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:35.957868 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d669e2cb-8772-4dc7-a1b2-7aa1b8194ff5 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:35.959196 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d669e2cb-8772-4dc7-a1b2-7aa1b8194ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:35.997267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614be4d5-cddc-4918-8a7b-55074dbc33de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:35.998179 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614be4d5-cddc-4918-8a7b-55074dbc33de None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:35.998467 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614be4d5-cddc-4918-8a7b-55074dbc33de None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:35.998752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614be4d5-cddc-4918-8a7b-55074dbc33de None admin] PATH_INFO: `/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:35.999961 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614be4d5-cddc-4918-8a7b-55074dbc33de None admin] RBAC: Authorizing `identity:get_project(project_id=c9f5c8ab86d44f42af60e2d8442a4f3d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:36.001432 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614be4d5-cddc-4918-8a7b-55074dbc33de None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:36.003351 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 100/199] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:50:35 2022] GET /identity/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:36.004047 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614be4d5-cddc-4918-8a7b-55074dbc33de None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:36.044551 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 100/200] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:36.045407 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d669e2cb-8772-4dc7-a1b2-7aa1b8194ff5 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:36.132001 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614be4d5-cddc-4918-8a7b-55074dbc33de None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:36.177211 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d2248-a2a3-486a-b8bf-f545e1869226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:36.177913 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d2248-a2a3-486a-b8bf-f545e1869226 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:36.178206 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d2248-a2a3-486a-b8bf-f545e1869226 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:36.178482 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d2248-a2a3-486a-b8bf-f545e1869226 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:36.179337 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27d2248-a2a3-486a-b8bf-f545e1869226 None admin] RBAC: Adding query filter params (role.id=85dcd5509fca4911881d45baf5f1d5a5, scope.project.id=c9f5c8ab86d44f42af60e2d8442a4f3d, user.id=a75fbca2c9d442b2bca8554b27dc944a) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:36.179508 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d669e2cb-8772-4dc7-a1b2-7aa1b8194ff5 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:36.179662 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27d2248-a2a3-486a-b8bf-f545e1869226 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:36.181116 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27d2248-a2a3-486a-b8bf-f545e1869226 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:36.187521 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 101/201] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:50:36 2022] GET /identity/v3/role_assignments?user.id=a75fbca2c9d442b2bca8554b27dc944a&scope.project.id=c9f5c8ab86d44f42af60e2d8442a4f3d&role.id=85dcd5509fca4911881d45baf5f1d5a5 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:36.188365 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d2248-a2a3-486a-b8bf-f545e1869226 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:36.192188 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:36.225332 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf988a42-e14a-4c78-b609-cf3e5144ed38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:36.226337 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf988a42-e14a-4c78-b609-cf3e5144ed38 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:36.226626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf988a42-e14a-4c78-b609-cf3e5144ed38 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:36.226908 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf988a42-e14a-4c78-b609-cf3e5144ed38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:36.227788 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf988a42-e14a-4c78-b609-cf3e5144ed38 None admin] RBAC: Adding query filter params (role.id=475521b8a0174037b162f92022aea1a1, scope.project.id=c9f5c8ab86d44f42af60e2d8442a4f3d, user.id=a75fbca2c9d442b2bca8554b27dc944a) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:36.228382 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf988a42-e14a-4c78-b609-cf3e5144ed38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:36.230029 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:36.231185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf988a42-e14a-4c78-b609-cf3e5144ed38 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:36.246298 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 101/202] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:50:36 2022] GET /identity/v3/role_assignments?user.id=a75fbca2c9d442b2bca8554b27dc944a&scope.project.id=c9f5c8ab86d44f42af60e2d8442a4f3d&role.id=475521b8a0174037b162f92022aea1a1 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:36.325547 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d2248-a2a3-486a-b8bf-f545e1869226 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:36.376343 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372cdab-813d-4004-9c0b-a02750d2ef5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:36.377424 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372cdab-813d-4004-9c0b-a02750d2ef5f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:36.377882 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372cdab-813d-4004-9c0b-a02750d2ef5f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:36.378376 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372cdab-813d-4004-9c0b-a02750d2ef5f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:36.392046 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e372cdab-813d-4004-9c0b-a02750d2ef5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:36.400992 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e372cdab-813d-4004-9c0b-a02750d2ef5f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 15:50:36.400992 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:36.400992 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:36.400992 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:36.400992 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:36.400992 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:36.400992 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:36.400992 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:36.400992 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:36.400992 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:36.400992 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:36.400992 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:36.400992 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:36.400992 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:36.400992 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:36.400992 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:36.402174 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:36.402174 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:36.402174 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:36.402174 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:36.402174 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:36.402174 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:36.402174 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:36.402174 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:36.402174 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:36.402174 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:36.402174 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:36.402174 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:36.402174 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:36.402174 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:36.402174 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:36.402174 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:36.403114 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:36.403114 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:36.403114 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:36.403114 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:36.403114 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:36.403114 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:36.403114 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:36.403114 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:36.403114 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:36.403114 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:36.403114 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 15:50:36.403114 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:50:36.405373 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 102/203] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:50:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:36.414812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf988a42-e14a-4c78-b609-cf3e5144ed38 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:36.583646 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf988a42-e14a-4c78-b609-cf3e5144ed38 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:36.639302 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0162de-0e77-4a7f-8484-fc1a5094f168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:36.640065 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0162de-0e77-4a7f-8484-fc1a5094f168 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:36.640355 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0162de-0e77-4a7f-8484-fc1a5094f168 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:36.640632 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0162de-0e77-4a7f-8484-fc1a5094f168 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:36.641470 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0162de-0e77-4a7f-8484-fc1a5094f168 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:36.641804 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0162de-0e77-4a7f-8484-fc1a5094f168 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:36.643234 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0162de-0e77-4a7f-8484-fc1a5094f168 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:36.655815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 102/204] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 15:50:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:36.661855 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372cdab-813d-4004-9c0b-a02750d2ef5f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:36.776432 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:36.786594 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372cdab-813d-4004-9c0b-a02750d2ef5f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:36.824656 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c603da25-4c2f-4cae-bcf3-4905f4b0fe72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:36.826496 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603da25-4c2f-4cae-bcf3-4905f4b0fe72 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:36.827015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603da25-4c2f-4cae-bcf3-4905f4b0fe72 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:36.827492 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603da25-4c2f-4cae-bcf3-4905f4b0fe72 None admin] PATH_INFO: `/v3/users/3418daf276a84936bf5896c885e85a1f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:36.829026 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c603da25-4c2f-4cae-bcf3-4905f4b0fe72 None admin] RBAC: Authorizing `identity:get_user(user_id=3418daf276a84936bf5896c885e85a1f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:36.830754 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c603da25-4c2f-4cae-bcf3-4905f4b0fe72 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:36.833203 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 103/205] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:36 2022] GET /identity/v3/users/3418daf276a84936bf5896c885e85a1f => generated 289 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:36.839453 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0162de-0e77-4a7f-8484-fc1a5094f168 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:36.926839 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:37.030762 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0162de-0e77-4a7f-8484-fc1a5094f168 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:37.082945 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597d05f-cec1-43e2-9763-e03fff2d35e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:37.083629 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597d05f-cec1-43e2-9763-e03fff2d35e4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:37.083919 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597d05f-cec1-43e2-9763-e03fff2d35e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:37.084193 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597d05f-cec1-43e2-9763-e03fff2d35e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:37.163299 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7597d05f-cec1-43e2-9763-e03fff2d35e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:37.164713 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7597d05f-cec1-43e2-9763-e03fff2d35e4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:37.211925 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:37.288339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 103/206] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:50:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 15:50:37.293800 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c603da25-4c2f-4cae-bcf3-4905f4b0fe72 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:37.469298 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c603da25-4c2f-4cae-bcf3-4905f4b0fe72 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:37.520715 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed47b98-22cd-4912-9739-dc5dcde95d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:37.521997 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed47b98-22cd-4912-9739-dc5dcde95d8e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:37.522509 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed47b98-22cd-4912-9739-dc5dcde95d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:37.522983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed47b98-22cd-4912-9739-dc5dcde95d8e None admin] PATH_INFO: `/v3/projects/8edda125d8bb41498eca65801bd10882` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:37.530327 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed47b98-22cd-4912-9739-dc5dcde95d8e None admin] RBAC: Authorizing `identity:get_project(project_id=8edda125d8bb41498eca65801bd10882)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:37.532051 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed47b98-22cd-4912-9739-dc5dcde95d8e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:37.534064 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 104/207] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:50:37 2022] GET /identity/v3/projects/8edda125d8bb41498eca65801bd10882 => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:37.540978 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597d05f-cec1-43e2-9763-e03fff2d35e4 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:37.689218 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597d05f-cec1-43e2-9763-e03fff2d35e4 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:37.730500 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:37.742913 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1fad2-bac3-4b63-b416-c23253ed1bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:37.744118 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1fad2-bac3-4b63-b416-c23253ed1bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:37.744845 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1fad2-bac3-4b63-b416-c23253ed1bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:37.745129 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1fad2-bac3-4b63-b416-c23253ed1bc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:37.746025 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f1fad2-bac3-4b63-b416-c23253ed1bc9 None admin] RBAC: Adding query filter params (role.id=f4d0b371e4b140d9a9f6d9c6983c0ec8, scope.project.id=8edda125d8bb41498eca65801bd10882, user.id=3418daf276a84936bf5896c885e85a1f) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:37.746400 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f1fad2-bac3-4b63-b416-c23253ed1bc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:37.747882 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f1fad2-bac3-4b63-b416-c23253ed1bc9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:37.754966 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 104/208] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:50:37 2022] GET /identity/v3/role_assignments?user.id=3418daf276a84936bf5896c885e85a1f&scope.project.id=8edda125d8bb41498eca65801bd10882&role.id=f4d0b371e4b140d9a9f6d9c6983c0ec8 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:38.403145 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619d297-b914-461f-96d3-6d131835f0b9 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:38.404155 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619d297-b914-461f-96d3-6d131835f0b9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:38.404653 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619d297-b914-461f-96d3-6d131835f0b9 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:38.406588 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 105/209] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:38.412065 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16527b43-bba9-4e33-ad35-56155695ef83 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:38.412899 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16527b43-bba9-4e33-ad35-56155695ef83 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:38.413382 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16527b43-bba9-4e33-ad35-56155695ef83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:38.430650 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-16527b43-bba9-4e33-ad35-56155695ef83 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:38.432180 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-16527b43-bba9-4e33-ad35-56155695ef83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:38.507736 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 105/210] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:38.530964 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b619d297-b914-461f-96d3-6d131835f0b9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:38.560933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:38.669371 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b619d297-b914-461f-96d3-6d131835f0b9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:38.707323 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3312a644-2f12-4d45-bceb-83a4498249f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:38.708142 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312a644-2f12-4d45-bceb-83a4498249f0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:38.708430 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312a644-2f12-4d45-bceb-83a4498249f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:38.708710 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312a644-2f12-4d45-bceb-83a4498249f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:38.720390 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3312a644-2f12-4d45-bceb-83a4498249f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:38.729199 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-3312a644-2f12-4d45-bceb-83a4498249f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:50:38.729199 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:38.729199 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:38.729199 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:38.729199 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:38.729199 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:38.729199 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:38.729199 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:38.729199 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:38.729199 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:38.729199 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:38.729199 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:38.729199 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:38.729199 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:38.729199 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:38.729199 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:38.730201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:38.730201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:38.730201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:38.730201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:38.730201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:38.730201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:38.730201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:38.730201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:38.730201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:38.730201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:38.730201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:38.730201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:38.730201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:38.730201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:38.730201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:38.730201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:38.731027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:38.731027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:38.731027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:38.731027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:38.731027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:38.731027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:38.731027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:38.731027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:38.731027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:38.731027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:38.731027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:50:38.731027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:50:38.731027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 106/211] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:50:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:38.731694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d835c4d-1755-4323-934d-8104882b9cfe None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:38.731694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d835c4d-1755-4323-934d-8104882b9cfe None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:38.731819 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d835c4d-1755-4323-934d-8104882b9cfe None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:38.733106 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 106/212] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:38.740432 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3312a644-2f12-4d45-bceb-83a4498249f0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:38.742689 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028edad0-b381-4b36-9485-c7e9edaed576 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:38.744125 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028edad0-b381-4b36-9485-c7e9edaed576 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:38.744628 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028edad0-b381-4b36-9485-c7e9edaed576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:38.768546 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-028edad0-b381-4b36-9485-c7e9edaed576 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:38.770522 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-028edad0-b381-4b36-9485-c7e9edaed576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:38.860230 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:38.863987 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 107/213] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 15:50:38.879547 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3312a644-2f12-4d45-bceb-83a4498249f0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:38.889441 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028edad0-b381-4b36-9485-c7e9edaed576 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:38.921006 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a4369-618e-42c5-8f85-575178c2b47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:38.922267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a4369-618e-42c5-8f85-575178c2b47f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:38.922710 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a4369-618e-42c5-8f85-575178c2b47f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:38.923271 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a4369-618e-42c5-8f85-575178c2b47f None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:38.925370 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a4369-618e-42c5-8f85-575178c2b47f None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:38.925912 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a4369-618e-42c5-8f85-575178c2b47f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:38.927661 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a4369-618e-42c5-8f85-575178c2b47f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:38.939290 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 107/214] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 15:50:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:38.947136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c785a98-b33b-4840-b7dd-165ba1a8b9cf None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:38.947651 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c785a98-b33b-4840-b7dd-165ba1a8b9cf None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:38.948737 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c785a98-b33b-4840-b7dd-165ba1a8b9cf None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:38.950090 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 108/215] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:38.950638 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c785a98-b33b-4840-b7dd-165ba1a8b9cf None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:38.957878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:39.006568 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028edad0-b381-4b36-9485-c7e9edaed576 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:39.047621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e0432-85b5-40f7-9199-d50cb98fcebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:39.048431 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e0432-85b5-40f7-9199-d50cb98fcebb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:39.048724 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e0432-85b5-40f7-9199-d50cb98fcebb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:39.049001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e0432-85b5-40f7-9199-d50cb98fcebb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:39.050486 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e0432-85b5-40f7-9199-d50cb98fcebb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:39.052153 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e0432-85b5-40f7-9199-d50cb98fcebb None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:39.056892 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 108/216] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:50:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:50:39.056892 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542556a6-e1de-4894-9f64-2ad78f94654e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:39.056892 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542556a6-e1de-4894-9f64-2ad78f94654e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:39.056892 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542556a6-e1de-4894-9f64-2ad78f94654e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:39.075299 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-542556a6-e1de-4894-9f64-2ad78f94654e None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:39.076755 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c785a98-b33b-4840-b7dd-165ba1a8b9cf None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:39.076871 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-542556a6-e1de-4894-9f64-2ad78f94654e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:39.115091 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4612e56-644d-4b72-8abf-be461458f109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:39.115970 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4612e56-644d-4b72-8abf-be461458f109 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:39.116257 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4612e56-644d-4b72-8abf-be461458f109 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:39.116536 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4612e56-644d-4b72-8abf-be461458f109 None admin] PATH_INFO: `/v3/users/19a547136f6d40e097f7fbe83286fc89` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:39.130819 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4612e56-644d-4b72-8abf-be461458f109 None admin] RBAC: Authorizing `identity:get_user(user_id=19a547136f6d40e097f7fbe83286fc89)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:39.132307 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4612e56-644d-4b72-8abf-be461458f109 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:39.134102 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 109/217] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:38 2022] GET /identity/v3/users/19a547136f6d40e097f7fbe83286fc89 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:39.138732 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e85a2e-33a1-4b04-97e2-e546b8663d04 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:39.138732 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e85a2e-33a1-4b04-97e2-e546b8663d04 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:39.138732 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e85a2e-33a1-4b04-97e2-e546b8663d04 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:39.138732 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 110/218] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:39.138732 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e85a2e-33a1-4b04-97e2-e546b8663d04 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:39.149052 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:39.154871 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 109/219] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:39.161801 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542556a6-e1de-4894-9f64-2ad78f94654e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:39.218453 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:39.254748 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e85a2e-33a1-4b04-97e2-e546b8663d04 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:39.265492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542556a6-e1de-4894-9f64-2ad78f94654e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:39.289591 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd591fd-8562-4527-a364-f6c1fb7f9ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:39.290279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd591fd-8562-4527-a364-f6c1fb7f9ef5 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:39.290565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd591fd-8562-4527-a364-f6c1fb7f9ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:39.290845 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd591fd-8562-4527-a364-f6c1fb7f9ef5 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:39.291568 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd591fd-8562-4527-a364-f6c1fb7f9ef5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:39.293045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd591fd-8562-4527-a364-f6c1fb7f9ef5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:39.299436 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8917944-fc00-4b33-a27e-7be2fcecd984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:39.300509 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8917944-fc00-4b33-a27e-7be2fcecd984 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:39.300790 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8917944-fc00-4b33-a27e-7be2fcecd984 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:39.301063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8917944-fc00-4b33-a27e-7be2fcecd984 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:39.315471 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 111/220] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 15:50:39 2022] POST /identity/v3/users => generated 297 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:50:39.317301 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e051df-7b7a-4e31-a370-01ea7419c5c2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:39.317603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e051df-7b7a-4e31-a370-01ea7419c5c2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:39.317897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e051df-7b7a-4e31-a370-01ea7419c5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:39.348058 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e4e051df-7b7a-4e31-a370-01ea7419c5c2 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:39.349564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e4e051df-7b7a-4e31-a370-01ea7419c5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:39.391612 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8917944-fc00-4b33-a27e-7be2fcecd984 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:39.393192 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8917944-fc00-4b33-a27e-7be2fcecd984 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:39.417098 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:39.430116 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 112/221] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:39.437563 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e051df-7b7a-4e31-a370-01ea7419c5c2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:39.467355 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:39.530840 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 110/222] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:50:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 15:50:39.531978 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8917944-fc00-4b33-a27e-7be2fcecd984 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:39.582633 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e051df-7b7a-4e31-a370-01ea7419c5c2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:39.632689 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19059f9-d312-4d92-a849-cb34236743d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:39.633525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19059f9-d312-4d92-a849-cb34236743d0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:39.634203 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19059f9-d312-4d92-a849-cb34236743d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:39.634489 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19059f9-d312-4d92-a849-cb34236743d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:39.648599 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19059f9-d312-4d92-a849-cb34236743d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:39.656515 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d19059f9-d312-4d92-a849-cb34236743d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:50:39.656515 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:39.656515 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:39.656515 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:39.656515 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:39.656515 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:39.656515 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:39.656515 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:39.656515 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:39.656515 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:39.656515 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:39.656515 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:39.656515 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:39.656515 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:39.656515 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:39.656515 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:39.657493 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:39.657493 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:39.657493 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:39.657493 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:39.657493 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:39.657493 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:39.657493 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:39.657493 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:39.657493 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:39.657493 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:39.657493 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:39.657493 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:39.657493 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:39.657493 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:39.657493 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:39.657493 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:39.658352 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:39.658352 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:39.658352 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:39.658352 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:39.658352 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:39.658352 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:39.658352 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:39.658352 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:39.658352 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:39.658352 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:39.658352 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:50:39.658352 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:50:39.658968 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8917944-fc00-4b33-a27e-7be2fcecd984 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:39.659040 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 113/223] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:50:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:39.660158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19059f9-d312-4d92-a849-cb34236743d0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:39.706053 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba328a0-248c-4ab0-a117-592aa363ee13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:39.707076 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba328a0-248c-4ab0-a117-592aa363ee13 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:39.707425 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba328a0-248c-4ab0-a117-592aa363ee13 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:39.707767 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba328a0-248c-4ab0-a117-592aa363ee13 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:39.724240 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba328a0-248c-4ab0-a117-592aa363ee13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:39.731617 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:39.740498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-fba328a0-248c-4ab0-a117-592aa363ee13 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 15:50:39.740498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:39.740498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:39.740498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:39.740498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:39.740498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:39.740498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:39.740498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:39.740498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:39.740498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:39.740498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:39.740498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:39.740498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:39.740498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:39.740498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:39.740498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:39.742520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:39.742520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:39.742520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:39.742520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:39.742520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:39.742520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:39.742520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:39.742520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:39.742520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:39.742520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:39.742520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:39.742520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:39.742520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:39.742520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:39.742520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:39.742520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:39.743738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:39.743738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:39.743738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:39.743738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:39.743738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:39.743738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:39.743738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:39.743738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:39.743738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:39.743738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:39.743738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 15:50:39.743738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:50:39.743738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 111/224] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 15:50:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:39.745173 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba328a0-248c-4ab0-a117-592aa363ee13 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:39.768452 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:39.790443 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19059f9-d312-4d92-a849-cb34236743d0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:39.828753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59418faf-b796-41e4-b0d6-e789b5c36297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:39.829170 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59418faf-b796-41e4-b0d6-e789b5c36297 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:39.829451 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59418faf-b796-41e4-b0d6-e789b5c36297 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:39.829755 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59418faf-b796-41e4-b0d6-e789b5c36297 None admin] PATH_INFO: `/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:39.831532 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59418faf-b796-41e4-b0d6-e789b5c36297 None admin] RBAC: Authorizing `identity:get_project(project_id=c9f5c8ab86d44f42af60e2d8442a4f3d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:39.833180 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59418faf-b796-41e4-b0d6-e789b5c36297 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:39.834974 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 114/225] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:50:39 2022] GET /identity/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:39.835590 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59418faf-b796-41e4-b0d6-e789b5c36297 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:39.869597 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba328a0-248c-4ab0-a117-592aa363ee13 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:39.912099 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a28ffea-7cd8-4909-a4ce-268837ab6de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:39.913206 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a28ffea-7cd8-4909-a4ce-268837ab6de3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:39.913710 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a28ffea-7cd8-4909-a4ce-268837ab6de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:39.914501 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a28ffea-7cd8-4909-a4ce-268837ab6de3 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:39.915515 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a28ffea-7cd8-4909-a4ce-268837ab6de3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:39.916009 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a28ffea-7cd8-4909-a4ce-268837ab6de3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:39.917657 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a28ffea-7cd8-4909-a4ce-268837ab6de3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:39.927786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 112/226] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:50:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:39.933762 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a28ffea-7cd8-4909-a4ce-268837ab6de3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:39.950288 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59418faf-b796-41e4-b0d6-e789b5c36297 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:39.974845 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:39.994850 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1fb063-2a1e-4e46-af2c-9ddfbd435071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:39.996245 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fb063-2a1e-4e46-af2c-9ddfbd435071 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:39.996669 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fb063-2a1e-4e46-af2c-9ddfbd435071 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:39.997065 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fb063-2a1e-4e46-af2c-9ddfbd435071 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:39.998520 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1fb063-2a1e-4e46-af2c-9ddfbd435071 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:39.999007 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1fb063-2a1e-4e46-af2c-9ddfbd435071 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:40.001337 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1fb063-2a1e-4e46-af2c-9ddfbd435071 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:40.016229 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 115/227] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:50:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:40.016924 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1fb063-2a1e-4e46-af2c-9ddfbd435071 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:40.032943 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:40.051810 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a28ffea-7cd8-4909-a4ce-268837ab6de3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:40.090309 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed74ce2d-d551-44c4-a1a4-6cef635604c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:40.091400 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74ce2d-d551-44c4-a1a4-6cef635604c0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:40.091886 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74ce2d-d551-44c4-a1a4-6cef635604c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:40.092407 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74ce2d-d551-44c4-a1a4-6cef635604c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:40.093450 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed74ce2d-d551-44c4-a1a4-6cef635604c0 None admin] RBAC: Adding query filter params (role.id=54195e660fc349978b39d5794765a095, scope.project.id=c9f5c8ab86d44f42af60e2d8442a4f3d, user.id=19a547136f6d40e097f7fbe83286fc89) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:40.094005 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed74ce2d-d551-44c4-a1a4-6cef635604c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:40.095887 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed74ce2d-d551-44c4-a1a4-6cef635604c0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:40.102742 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 113/228] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:50:39 2022] GET /identity/v3/role_assignments?user.id=19a547136f6d40e097f7fbe83286fc89&scope.project.id=c9f5c8ab86d44f42af60e2d8442a4f3d&role.id=54195e660fc349978b39d5794765a095 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:40.103878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed74ce2d-d551-44c4-a1a4-6cef635604c0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:40.152523 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1fb063-2a1e-4e46-af2c-9ddfbd435071 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:40.198141 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5722697-0af4-4a11-bc1a-3642aa607d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:40.200151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5722697-0af4-4a11-bc1a-3642aa607d60 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:40.200917 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5722697-0af4-4a11-bc1a-3642aa607d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:40.201681 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5722697-0af4-4a11-bc1a-3642aa607d60 None admin] PATH_INFO: `/v3/users/a75fbca2c9d442b2bca8554b27dc944a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:40.205062 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5722697-0af4-4a11-bc1a-3642aa607d60 None admin] RBAC: Authorizing `identity:get_user(user_id=a75fbca2c9d442b2bca8554b27dc944a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:40.209300 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5722697-0af4-4a11-bc1a-3642aa607d60 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:40.214002 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 116/229] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:40 2022] GET /identity/v3/users/a75fbca2c9d442b2bca8554b27dc944a => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:40.224480 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5722697-0af4-4a11-bc1a-3642aa607d60 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:40.349603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:40.368692 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed74ce2d-d551-44c4-a1a4-6cef635604c0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:40.419918 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a4e94-f671-4530-a411-f9fc8bab3925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:40.422480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a4e94-f671-4530-a411-f9fc8bab3925 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:40.423093 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a4e94-f671-4530-a411-f9fc8bab3925 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:40.423644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a4e94-f671-4530-a411-f9fc8bab3925 None admin] PATH_INFO: `/v3/users/a75fbca2c9d442b2bca8554b27dc944a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:40.425374 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a4e94-f671-4530-a411-f9fc8bab3925 None admin] RBAC: Authorizing `identity:get_user(user_id=a75fbca2c9d442b2bca8554b27dc944a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:40.427541 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a4e94-f671-4530-a411-f9fc8bab3925 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:40.429526 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 114/230] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:40 2022] GET /identity/v3/users/a75fbca2c9d442b2bca8554b27dc944a => generated 283 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:40.435901 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a4e94-f671-4530-a411-f9fc8bab3925 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:40.453309 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5722697-0af4-4a11-bc1a-3642aa607d60 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:40.465612 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:40.521917 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819f0ed-1035-4cbb-8e55-3b49ee1cbac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:40.524194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819f0ed-1035-4cbb-8e55-3b49ee1cbac3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:40.524712 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819f0ed-1035-4cbb-8e55-3b49ee1cbac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:40.525190 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819f0ed-1035-4cbb-8e55-3b49ee1cbac3 None admin] PATH_INFO: `/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:40.526727 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9819f0ed-1035-4cbb-8e55-3b49ee1cbac3 None admin] RBAC: Authorizing `identity:get_project(project_id=c9f5c8ab86d44f42af60e2d8442a4f3d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:40.528964 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9819f0ed-1035-4cbb-8e55-3b49ee1cbac3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:40.531379 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 117/231] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:50:40 2022] GET /identity/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d => generated 302 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:40.538528 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819f0ed-1035-4cbb-8e55-3b49ee1cbac3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:40.661424 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819f0ed-1035-4cbb-8e55-3b49ee1cbac3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:40.686316 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:40.704208 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8587d42-302e-4323-9ee0-1d89559e372d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:40.706445 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8587d42-302e-4323-9ee0-1d89559e372d None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:40.706948 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8587d42-302e-4323-9ee0-1d89559e372d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:40.707426 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8587d42-302e-4323-9ee0-1d89559e372d None admin] PATH_INFO: `/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d/users/a75fbca2c9d442b2bca8554b27dc944a/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:40.710280 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8587d42-302e-4323-9ee0-1d89559e372d None admin] RBAC: Authorizing `identity:create_grant(project_id=c9f5c8ab86d44f42af60e2d8442a4f3d, user_id=a75fbca2c9d442b2bca8554b27dc944a, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:40.712090 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8587d42-302e-4323-9ee0-1d89559e372d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:40.712647 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a4e94-f671-4530-a411-f9fc8bab3925 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:40.732333 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 118/232] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 15:50:40 2022] PUT /identity/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d/users/a75fbca2c9d442b2bca8554b27dc944a/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 15:50:40.786433 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c633b17c-f177-491e-a3b8-3273c3f298eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:40.787387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633b17c-f177-491e-a3b8-3273c3f298eb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:40.787679 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633b17c-f177-491e-a3b8-3273c3f298eb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:40.788014 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633b17c-f177-491e-a3b8-3273c3f298eb None admin] PATH_INFO: `/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:40.791743 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c633b17c-f177-491e-a3b8-3273c3f298eb None admin] RBAC: Authorizing `identity:get_project(project_id=c9f5c8ab86d44f42af60e2d8442a4f3d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:40.793302 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c633b17c-f177-491e-a3b8-3273c3f298eb None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:40.795622 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 115/233] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:50:40 2022] GET /identity/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d => generated 302 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:40.802385 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8587d42-302e-4323-9ee0-1d89559e372d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:40.935844 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8587d42-302e-4323-9ee0-1d89559e372d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:40.974755 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547014a9-f226-4ced-b61b-c1dbb1e46a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:40.976178 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547014a9-f226-4ced-b61b-c1dbb1e46a0f None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:40.976699 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547014a9-f226-4ced-b61b-c1dbb1e46a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:40.977172 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547014a9-f226-4ced-b61b-c1dbb1e46a0f None admin] PATH_INFO: `/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d/users/a75fbca2c9d442b2bca8554b27dc944a/roles/85dcd5509fca4911881d45baf5f1d5a5` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:40.980080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547014a9-f226-4ced-b61b-c1dbb1e46a0f None admin] RBAC: Authorizing `identity:create_grant(project_id=c9f5c8ab86d44f42af60e2d8442a4f3d, user_id=a75fbca2c9d442b2bca8554b27dc944a, role_id=85dcd5509fca4911881d45baf5f1d5a5)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:40.981885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547014a9-f226-4ced-b61b-c1dbb1e46a0f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:41.005636 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 119/234] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 15:50:40 2022] PUT /identity/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d/users/a75fbca2c9d442b2bca8554b27dc944a/roles/85dcd5509fca4911881d45baf5f1d5a5 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 15:50:41.898727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc3a2c-4205-40fe-bb42-cda8273e138e None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:41.899250 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc3a2c-4205-40fe-bb42-cda8273e138e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:41.899361 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc3a2c-4205-40fe-bb42-cda8273e138e None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:41.900941 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 116/235] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:41.908561 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac84c6-3c97-46df-884a-306d2767b4cf None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:41.908561 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac84c6-3c97-46df-884a-306d2767b4cf None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:41.908561 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac84c6-3c97-46df-884a-306d2767b4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:41.930633 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-48ac84c6-3c97-46df-884a-306d2767b4cf None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:41.932173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-48ac84c6-3c97-46df-884a-306d2767b4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:41.951380 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc768a-4a07-498e-bd1d-4d0a73cfcbd5 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:41.951689 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc768a-4a07-498e-bd1d-4d0a73cfcbd5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:41.951952 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc768a-4a07-498e-bd1d-4d0a73cfcbd5 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:41.953453 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 117/236] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:41.962037 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c4f0d-25f9-438a-8f54-7eaf2772e3ce None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:41.962669 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c4f0d-25f9-438a-8f54-7eaf2772e3ce None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:41.963201 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c4f0d-25f9-438a-8f54-7eaf2772e3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:41.986083 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-318c4f0d-25f9-438a-8f54-7eaf2772e3ce None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:41.987838 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-318c4f0d-25f9-438a-8f54-7eaf2772e3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:41.998111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:42.064571 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 120/237] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:42.098903 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac84c6-3c97-46df-884a-306d2767b4cf None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:42.127206 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 118/238] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 15:50:42.165923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c4f0d-25f9-438a-8f54-7eaf2772e3ce None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:42.177567 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:42.230146 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac84c6-3c97-46df-884a-306d2767b4cf None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:42.279390 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900fd130-aaf2-4cf0-a818-c6fd45acf41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:42.280216 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fd130-aaf2-4cf0-a818-c6fd45acf41f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:42.280506 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fd130-aaf2-4cf0-a818-c6fd45acf41f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:42.280786 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fd130-aaf2-4cf0-a818-c6fd45acf41f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:42.283230 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900fd130-aaf2-4cf0-a818-c6fd45acf41f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:42.285299 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900fd130-aaf2-4cf0-a818-c6fd45acf41f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:42.287027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 121/239] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:50:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:50:42.293478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900fd130-aaf2-4cf0-a818-c6fd45acf41f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:42.386142 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c4f0d-25f9-438a-8f54-7eaf2772e3ce None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:42.400106 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:42.483547 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13136f58-39cb-4fdc-9dd8-19bcd7a696f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:42.484622 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13136f58-39cb-4fdc-9dd8-19bcd7a696f1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:42.484912 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13136f58-39cb-4fdc-9dd8-19bcd7a696f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:42.485191 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13136f58-39cb-4fdc-9dd8-19bcd7a696f1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:42.517723 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900fd130-aaf2-4cf0-a818-c6fd45acf41f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:42.522223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13136f58-39cb-4fdc-9dd8-19bcd7a696f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:42.531185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-13136f58-39cb-4fdc-9dd8-19bcd7a696f1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 15:50:42.531185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:42.531185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:42.531185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:42.531185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:42.531185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:42.531185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:42.531185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:42.531185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:42.531185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:42.531185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:42.531185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:42.531185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:42.531185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:42.531185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:42.531185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:42.532155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:42.532155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:42.532155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:42.532155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:42.532155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:42.532155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:42.532155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:42.532155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:42.532155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:42.532155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:42.532155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:42.532155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:42.532155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:42.532155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:42.532155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:42.532155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:42.532976 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:42.532976 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:42.532976 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:42.532976 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:42.532976 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:42.532976 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:42.532976 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:42.532976 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:42.532976 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:42.532976 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:42.532976 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 15:50:42.532976 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:50:42.532976 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 119/240] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:50:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:50:42.538467 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13136f58-39cb-4fdc-9dd8-19bcd7a696f1 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:42.557544 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201fa9b-7cca-4aba-9847-31b294043999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:42.558257 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201fa9b-7cca-4aba-9847-31b294043999 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:42.558741 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201fa9b-7cca-4aba-9847-31b294043999 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:42.559182 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201fa9b-7cca-4aba-9847-31b294043999 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:42.560545 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d201fa9b-7cca-4aba-9847-31b294043999 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:42.564358 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d201fa9b-7cca-4aba-9847-31b294043999 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:42.594542 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 122/241] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 15:50:42 2022] POST /identity/v3/users => generated 293 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:50:42.700863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:42.750948 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13136f58-39cb-4fdc-9dd8-19bcd7a696f1 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:42.788800 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ed027-d02d-4bb5-99e6-86f44a7d40d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:42.789486 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ed027-d02d-4bb5-99e6-86f44a7d40d1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:42.789805 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ed027-d02d-4bb5-99e6-86f44a7d40d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:42.790090 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ed027-d02d-4bb5-99e6-86f44a7d40d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:42.791163 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650ed027-d02d-4bb5-99e6-86f44a7d40d1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:42.791484 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650ed027-d02d-4bb5-99e6-86f44a7d40d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:42.792941 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650ed027-d02d-4bb5-99e6-86f44a7d40d1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:42.803641 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 120/242] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 15:50:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:42.809048 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201fa9b-7cca-4aba-9847-31b294043999 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:42.822149 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4cb5c-3743-42cd-b938-89ee879971f0 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:42.822452 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4cb5c-3743-42cd-b938-89ee879971f0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:42.822723 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4cb5c-3743-42cd-b938-89ee879971f0 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:42.824737 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 121/243] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:42.833908 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e593d-1fe7-4c2d-b16a-21a6ab1bd24a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:42.834205 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e593d-1fe7-4c2d-b16a-21a6ab1bd24a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:42.834481 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e593d-1fe7-4c2d-b16a-21a6ab1bd24a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:42.854087 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5a8e593d-1fe7-4c2d-b16a-21a6ab1bd24a None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:42.855407 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5a8e593d-1fe7-4c2d-b16a-21a6ab1bd24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:42.945953 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 122/244] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:42.972368 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:42.974687 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e593d-1fe7-4c2d-b16a-21a6ab1bd24a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:42.991375 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201fa9b-7cca-4aba-9847-31b294043999 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:43.036949 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a50039-5bb1-4e5f-9b4c-e90d50ed9fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:43.037838 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a50039-5bb1-4e5f-9b4c-e90d50ed9fbe None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:43.038161 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a50039-5bb1-4e5f-9b4c-e90d50ed9fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:43.038437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a50039-5bb1-4e5f-9b4c-e90d50ed9fbe None admin] PATH_INFO: `/v3/users/1e906c799a234414953cdb2cf27362a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:43.048454 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:43.058128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a50039-5bb1-4e5f-9b4c-e90d50ed9fbe None admin] RBAC: Authorizing `identity:get_user(user_id=1e906c799a234414953cdb2cf27362a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:43.059632 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a50039-5bb1-4e5f-9b4c-e90d50ed9fbe None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:43.066204 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 123/245] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:42 2022] GET /identity/v3/users/1e906c799a234414953cdb2cf27362a1 => generated 297 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:43.067194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a50039-5bb1-4e5f-9b4c-e90d50ed9fbe None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:43.107100 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e593d-1fe7-4c2d-b16a-21a6ab1bd24a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:43.146634 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8527f29-c09b-43f3-8378-69777bc9726d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:43.147924 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8527f29-c09b-43f3-8378-69777bc9726d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:43.148418 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8527f29-c09b-43f3-8378-69777bc9726d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:43.148897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8527f29-c09b-43f3-8378-69777bc9726d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:43.170350 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8527f29-c09b-43f3-8378-69777bc9726d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:43.181242 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-e8527f29-c09b-43f3-8378-69777bc9726d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:50:43.181242 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:43.181242 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:43.181242 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:43.181242 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:43.181242 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:43.181242 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:43.181242 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:43.181242 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:43.181242 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:43.181242 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:43.181242 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:43.181242 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:43.181242 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:43.181242 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:43.181242 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:43.182333 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:43.182333 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:43.182333 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:43.182333 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:43.182333 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:43.182333 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:43.182333 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:43.182333 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:43.182333 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:43.182333 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:43.182333 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:43.182333 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:43.182333 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:43.182333 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:43.182333 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:43.182333 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:43.183182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:43.183182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:43.183182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:43.183182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:43.183182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:43.183182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:43.183182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:43.183182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:43.183182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:43.183182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:43.183182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:50:43.183182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:50:43.185999 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 123/246] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:50:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:43.200161 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8527f29-c09b-43f3-8378-69777bc9726d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:43.227306 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a50039-5bb1-4e5f-9b4c-e90d50ed9fbe None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:43.277049 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a70ebc-6448-4593-9b19-902b07256a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:43.277778 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a70ebc-6448-4593-9b19-902b07256a7c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:43.278061 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a70ebc-6448-4593-9b19-902b07256a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:43.278335 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a70ebc-6448-4593-9b19-902b07256a7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:43.324765 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:43.356068 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8527f29-c09b-43f3-8378-69777bc9726d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:43.382327 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:43.387867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a70ebc-6448-4593-9b19-902b07256a7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:43.389367 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a70ebc-6448-4593-9b19-902b07256a7c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:43.411974 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01421b0e-9d76-43d9-a377-d87811cf66a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:43.413134 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01421b0e-9d76-43d9-a377-d87811cf66a9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:43.413650 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01421b0e-9d76-43d9-a377-d87811cf66a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:43.414130 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01421b0e-9d76-43d9-a377-d87811cf66a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:43.415204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01421b0e-9d76-43d9-a377-d87811cf66a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:43.415723 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01421b0e-9d76-43d9-a377-d87811cf66a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:43.417354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01421b0e-9d76-43d9-a377-d87811cf66a9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:43.437111 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 124/247] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 15:50:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:43.444292 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01421b0e-9d76-43d9-a377-d87811cf66a9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:43.515798 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 124/248] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:50:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 15:50:43.521634 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a70ebc-6448-4593-9b19-902b07256a7c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:43.578977 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:43.633060 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a70ebc-6448-4593-9b19-902b07256a7c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:43.666757 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01421b0e-9d76-43d9-a377-d87811cf66a9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:43.687836 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a93df5-5723-444e-a601-8613e9dfd8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:43.688804 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a93df5-5723-444e-a601-8613e9dfd8bf None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:43.689086 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a93df5-5723-444e-a601-8613e9dfd8bf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:43.689784 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a93df5-5723-444e-a601-8613e9dfd8bf None admin] PATH_INFO: `/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:43.691396 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a93df5-5723-444e-a601-8613e9dfd8bf None admin] RBAC: Authorizing `identity:get_project(project_id=c9f5c8ab86d44f42af60e2d8442a4f3d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:43.692988 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a93df5-5723-444e-a601-8613e9dfd8bf None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:43.695748 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 125/249] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:50:43 2022] GET /identity/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:43.697037 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26095c3a-4457-4b78-81fe-a90348955e26 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:43.698170 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26095c3a-4457-4b78-81fe-a90348955e26 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:43.699298 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26095c3a-4457-4b78-81fe-a90348955e26 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:43.701280 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 126/250] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:43.707671 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26095c3a-4457-4b78-81fe-a90348955e26 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:43.799885 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a3492e-b843-43be-943f-8902f206efd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:43.801380 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3492e-b843-43be-943f-8902f206efd1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:43.804011 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3492e-b843-43be-943f-8902f206efd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:43.804519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3492e-b843-43be-943f-8902f206efd1 None admin] PATH_INFO: `/v3/users/19a547136f6d40e097f7fbe83286fc89` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:43.807089 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a3492e-b843-43be-943f-8902f206efd1 None admin] RBAC: Authorizing `identity:get_user(user_id=19a547136f6d40e097f7fbe83286fc89)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:43.810951 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a3492e-b843-43be-943f-8902f206efd1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:43.820591 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 125/251] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:43 2022] GET /identity/v3/users/19a547136f6d40e097f7fbe83286fc89 => generated 297 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:43.824207 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30384bd6-70c5-49b4-aaa0-1ca8ef62b819 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:43.825452 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30384bd6-70c5-49b4-aaa0-1ca8ef62b819 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:43.828107 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30384bd6-70c5-49b4-aaa0-1ca8ef62b819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:43.877857 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-30384bd6-70c5-49b4-aaa0-1ca8ef62b819 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:43.879376 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-30384bd6-70c5-49b4-aaa0-1ca8ef62b819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:43.907095 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26095c3a-4457-4b78-81fe-a90348955e26 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:43.986703 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:44.044757 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:44.063970 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d37379d-9dcb-4655-89bc-ca445ccd5111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:44.067913 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d37379d-9dcb-4655-89bc-ca445ccd5111 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:44.068203 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d37379d-9dcb-4655-89bc-ca445ccd5111 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:44.068481 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d37379d-9dcb-4655-89bc-ca445ccd5111 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:44.069349 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d37379d-9dcb-4655-89bc-ca445ccd5111 None admin] RBAC: Adding query filter params (role.id=f4d0b371e4b140d9a9f6d9c6983c0ec8, scope.project.id=c9f5c8ab86d44f42af60e2d8442a4f3d, user.id=1e906c799a234414953cdb2cf27362a1) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:44.078117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d37379d-9dcb-4655-89bc-ca445ccd5111 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:44.079979 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d37379d-9dcb-4655-89bc-ca445ccd5111 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:44.087685 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 127/252] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:50:43 2022] GET /identity/v3/role_assignments?user.id=1e906c799a234414953cdb2cf27362a1&scope.project.id=c9f5c8ab86d44f42af60e2d8442a4f3d&role.id=f4d0b371e4b140d9a9f6d9c6983c0ec8 => generated 259 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:44.093623 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d37379d-9dcb-4655-89bc-ca445ccd5111 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:44.158213 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 126/253] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:44.211119 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30384bd6-70c5-49b4-aaa0-1ca8ef62b819 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:44.278865 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d37379d-9dcb-4655-89bc-ca445ccd5111 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:44.366436 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0eb0e1-8522-4d10-b4ec-e9bf85d117f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:44.367343 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0eb0e1-8522-4d10-b4ec-e9bf85d117f4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:44.367634 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0eb0e1-8522-4d10-b4ec-e9bf85d117f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:44.367929 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0eb0e1-8522-4d10-b4ec-e9bf85d117f4 None admin] PATH_INFO: `/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:44.369289 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0eb0e1-8522-4d10-b4ec-e9bf85d117f4 None admin] RBAC: Authorizing `identity:get_project(project_id=c9f5c8ab86d44f42af60e2d8442a4f3d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:44.370859 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0eb0e1-8522-4d10-b4ec-e9bf85d117f4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:44.372707 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 128/254] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:50:44 2022] GET /identity/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d => generated 302 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:44.372854 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30384bd6-70c5-49b4-aaa0-1ca8ef62b819 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:44.374107 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d1751f-f14f-441b-9c19-7da52749fbf3 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:44.374400 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d1751f-f14f-441b-9c19-7da52749fbf3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:44.374686 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d1751f-f14f-441b-9c19-7da52749fbf3 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:44.376173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 129/255] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:44.382408 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d1751f-f14f-441b-9c19-7da52749fbf3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:44.419997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b9afa-4d24-4a9f-b158-ac5a207bd6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:44.421162 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b9afa-4d24-4a9f-b158-ac5a207bd6b6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:44.421455 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b9afa-4d24-4a9f-b158-ac5a207bd6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:44.421767 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b9afa-4d24-4a9f-b158-ac5a207bd6b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:44.436943 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77b9afa-4d24-4a9f-b158-ac5a207bd6b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:44.447694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-f77b9afa-4d24-4a9f-b158-ac5a207bd6b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:50:44.447694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:44.447694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:44.447694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:44.447694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:44.447694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:44.447694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:44.447694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:44.447694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:44.447694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:44.447694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:44.447694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:44.447694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:44.447694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:44.447694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:44.447694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:44.448717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:44.448717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:44.448717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:44.448717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:44.448717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:44.448717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:44.448717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:44.448717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:44.448717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:44.448717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:44.448717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:44.448717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:44.448717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:44.448717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:44.448717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:44.448717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:44.454126 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:44.454126 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:44.454126 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:44.454126 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:44.454126 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:44.454126 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:44.454126 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:44.454126 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:44.454126 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:44.454126 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:44.454126 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:50:44.454126 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:50:44.454126 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 127/256] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:50:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:44.454126 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a33e33-f720-494c-8c39-b4cd0918421d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:44.462811 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a33e33-f720-494c-8c39-b4cd0918421d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:44.462811 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a33e33-f720-494c-8c39-b4cd0918421d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:44.495941 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d4a33e33-f720-494c-8c39-b4cd0918421d None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:44.498644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d4a33e33-f720-494c-8c39-b4cd0918421d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:44.505530 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:44.578100 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:44.614556 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d1751f-f14f-441b-9c19-7da52749fbf3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:44.664353 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7435d-9c64-4ada-8837-201d2ca14c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:44.665662 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7435d-9c64-4ada-8837-201d2ca14c1e None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:44.666198 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7435d-9c64-4ada-8837-201d2ca14c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:44.666681 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7435d-9c64-4ada-8837-201d2ca14c1e None admin] PATH_INFO: `/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d/users/19a547136f6d40e097f7fbe83286fc89/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:44.669099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab7435d-9c64-4ada-8837-201d2ca14c1e None admin] RBAC: Authorizing `identity:create_grant(project_id=c9f5c8ab86d44f42af60e2d8442a4f3d, user_id=19a547136f6d40e097f7fbe83286fc89, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:44.671096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab7435d-9c64-4ada-8837-201d2ca14c1e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:44.681090 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 128/257] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:44.686667 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a33e33-f720-494c-8c39-b4cd0918421d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:44.701471 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 130/258] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 15:50:44 2022] PUT /identity/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d/users/19a547136f6d40e097f7fbe83286fc89/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 15:50:44.796356 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7435d-9c64-4ada-8837-201d2ca14c1e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:44.863197 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a33e33-f720-494c-8c39-b4cd0918421d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:44.906116 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5533a70-68ed-44d3-9824-f1310caa43cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:44.907439 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5533a70-68ed-44d3-9824-f1310caa43cb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:44.907938 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5533a70-68ed-44d3-9824-f1310caa43cb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:44.908421 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5533a70-68ed-44d3-9824-f1310caa43cb None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:44.909637 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5533a70-68ed-44d3-9824-f1310caa43cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:44.910187 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5533a70-68ed-44d3-9824-f1310caa43cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:44.911979 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5533a70-68ed-44d3-9824-f1310caa43cb None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:44.925642 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 129/259] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:50:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:44.929265 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5533a70-68ed-44d3-9824-f1310caa43cb None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:44.936410 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7435d-9c64-4ada-8837-201d2ca14c1e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:44.953987 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:44.971360 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:44.996983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3feb1-a748-4eeb-8ee4-5f6f3ff6244a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:44.998204 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3feb1-a748-4eeb-8ee4-5f6f3ff6244a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:44.998658 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3feb1-a748-4eeb-8ee4-5f6f3ff6244a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:44.999110 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3feb1-a748-4eeb-8ee4-5f6f3ff6244a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:45.013616 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b3feb1-a748-4eeb-8ee4-5f6f3ff6244a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:45.022689 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-05b3feb1-a748-4eeb-8ee4-5f6f3ff6244a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 15:50:45.022689 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:45.022689 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:45.022689 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:45.022689 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:45.022689 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:45.022689 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:45.022689 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:45.022689 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:45.022689 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:45.022689 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:45.022689 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:45.022689 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:45.022689 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:45.022689 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:45.022689 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:45.023682 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:45.023682 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:45.023682 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:45.023682 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:45.023682 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:45.023682 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:45.023682 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:45.023682 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:45.023682 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:45.023682 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:45.023682 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:45.023682 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:45.023682 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:45.023682 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:45.023682 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:45.023682 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:45.025127 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:45.025127 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:45.025127 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:45.025127 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:45.025127 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:45.025127 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:45.025127 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:45.025127 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:45.025127 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:45.025127 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:45.025127 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 15:50:45.025127 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:50:45.025127 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 131/260] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 15:50:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:50:45.029759 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3feb1-a748-4eeb-8ee4-5f6f3ff6244a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:45.063778 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5533a70-68ed-44d3-9824-f1310caa43cb None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:45.105449 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3cc51-4bc9-4afb-862b-6ae7f4c8de49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:45.107458 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3cc51-4bc9-4afb-862b-6ae7f4c8de49 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:45.108189 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3cc51-4bc9-4afb-862b-6ae7f4c8de49 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:45.108907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3cc51-4bc9-4afb-862b-6ae7f4c8de49 None admin] PATH_INFO: `/v3/users/a75fbca2c9d442b2bca8554b27dc944a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:45.111324 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf3cc51-4bc9-4afb-862b-6ae7f4c8de49 None admin] RBAC: Authorizing `identity:get_user(user_id=a75fbca2c9d442b2bca8554b27dc944a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:45.114981 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf3cc51-4bc9-4afb-862b-6ae7f4c8de49 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:45.121627 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 130/261] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:44 2022] GET /identity/v3/users/a75fbca2c9d442b2bca8554b27dc944a => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:45.123665 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3cc51-4bc9-4afb-862b-6ae7f4c8de49 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:45.149238 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3feb1-a748-4eeb-8ee4-5f6f3ff6244a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:45.188179 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734f9bd-c210-4523-a743-95fea437c541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:45.188777 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734f9bd-c210-4523-a743-95fea437c541 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:45.189109 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734f9bd-c210-4523-a743-95fea437c541 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:45.189387 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734f9bd-c210-4523-a743-95fea437c541 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:45.190785 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4734f9bd-c210-4523-a743-95fea437c541 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:45.191094 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4734f9bd-c210-4523-a743-95fea437c541 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:45.192513 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4734f9bd-c210-4523-a743-95fea437c541 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:45.201735 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 132/262] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:50:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:50:45.207908 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734f9bd-c210-4523-a743-95fea437c541 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:45.249636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:45.249836 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:45.255797 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3cc51-4bc9-4afb-862b-6ae7f4c8de49 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:45.296319 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a54ef-d82d-4381-b097-12eb17b42909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:45.297102 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a54ef-d82d-4381-b097-12eb17b42909 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:45.297395 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a54ef-d82d-4381-b097-12eb17b42909 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:45.297732 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a54ef-d82d-4381-b097-12eb17b42909 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:45.320866 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734f9bd-c210-4523-a743-95fea437c541 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:45.358215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b1cb26-f780-447b-a267-86b6391a53af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:45.359099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1cb26-f780-447b-a267-86b6391a53af None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:45.359381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1cb26-f780-447b-a267-86b6391a53af None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:45.359656 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1cb26-f780-447b-a267-86b6391a53af None admin] PATH_INFO: `/v3/users/a75fbca2c9d442b2bca8554b27dc944a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:45.360903 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b1cb26-f780-447b-a267-86b6391a53af None admin] RBAC: Authorizing `identity:get_user(user_id=a75fbca2c9d442b2bca8554b27dc944a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:45.363222 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b1cb26-f780-447b-a267-86b6391a53af None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:45.365716 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 133/263] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:45 2022] GET /identity/v3/users/a75fbca2c9d442b2bca8554b27dc944a => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:45.371585 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b1cb26-f780-447b-a267-86b6391a53af None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:45.401010 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3a54ef-d82d-4381-b097-12eb17b42909 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:45.404027 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3a54ef-d82d-4381-b097-12eb17b42909 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:45.508902 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b1cb26-f780-447b-a267-86b6391a53af None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:45.511183 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:45.523891 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:45.530299 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 131/264] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:50:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 15:50:45.537164 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a54ef-d82d-4381-b097-12eb17b42909 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:45.555131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b171273-fc17-46e9-8590-8ce6d5fa8637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:45.556242 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b171273-fc17-46e9-8590-8ce6d5fa8637 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:45.556762 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b171273-fc17-46e9-8590-8ce6d5fa8637 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:45.557258 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b171273-fc17-46e9-8590-8ce6d5fa8637 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:45.634201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b171273-fc17-46e9-8590-8ce6d5fa8637 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:45.635847 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b171273-fc17-46e9-8590-8ce6d5fa8637 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:45.656523 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a54ef-d82d-4381-b097-12eb17b42909 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:45.701450 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da5641-0ba0-4716-97c1-706512eef11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:45.702679 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da5641-0ba0-4716-97c1-706512eef11c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:45.703115 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da5641-0ba0-4716-97c1-706512eef11c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:45.703570 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da5641-0ba0-4716-97c1-706512eef11c None admin] PATH_INFO: `/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:45.705159 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da5641-0ba0-4716-97c1-706512eef11c None admin] RBAC: Authorizing `identity:get_project(project_id=c9f5c8ab86d44f42af60e2d8442a4f3d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:45.707322 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da5641-0ba0-4716-97c1-706512eef11c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:45.709342 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 132/265] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:50:45 2022] GET /identity/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:45.715612 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da5641-0ba0-4716-97c1-706512eef11c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:45.751888 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 134/266] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:50:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 15:50:45.761693 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b171273-fc17-46e9-8590-8ce6d5fa8637 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:45.762473 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:45.845781 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:45.929128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b171273-fc17-46e9-8590-8ce6d5fa8637 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:45.931344 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da5641-0ba0-4716-97c1-706512eef11c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:45.965158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e298e1-b19f-4da7-855f-1c8ab24ca392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:45.966228 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e298e1-b19f-4da7-855f-1c8ab24ca392 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:45.966513 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e298e1-b19f-4da7-855f-1c8ab24ca392 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:45.966789 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e298e1-b19f-4da7-855f-1c8ab24ca392 None admin] PATH_INFO: `/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:45.968040 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e298e1-b19f-4da7-855f-1c8ab24ca392 None admin] RBAC: Authorizing `identity:get_project(project_id=c9f5c8ab86d44f42af60e2d8442a4f3d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:45.969523 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e298e1-b19f-4da7-855f-1c8ab24ca392 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:45.971483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 135/267] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:50:45 2022] GET /identity/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:50:45.977609 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e298e1-b19f-4da7-855f-1c8ab24ca392 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:46.046402 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ce59a-49d8-4df4-8696-2a0f3dd030f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:46.046997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ce59a-49d8-4df4-8696-2a0f3dd030f5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:46.047282 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ce59a-49d8-4df4-8696-2a0f3dd030f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:46.047560 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ce59a-49d8-4df4-8696-2a0f3dd030f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:46.048416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ce59a-49d8-4df4-8696-2a0f3dd030f5 None admin] RBAC: Adding query filter params (role.id=475521b8a0174037b162f92022aea1a1, scope.project.id=c9f5c8ab86d44f42af60e2d8442a4f3d, user.id=a75fbca2c9d442b2bca8554b27dc944a) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:46.048733 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ce59a-49d8-4df4-8696-2a0f3dd030f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:46.050765 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ce59a-49d8-4df4-8696-2a0f3dd030f5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:46.060668 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:46.072008 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 133/268] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:50:45 2022] GET /identity/v3/role_assignments?user.id=a75fbca2c9d442b2bca8554b27dc944a&scope.project.id=c9f5c8ab86d44f42af60e2d8442a4f3d&role.id=475521b8a0174037b162f92022aea1a1 => generated 608 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:50:46.105218 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e298e1-b19f-4da7-855f-1c8ab24ca392 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:46.153540 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a93a7c-5944-4d08-af23-6ea4f14c7a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:46.154286 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a93a7c-5944-4d08-af23-6ea4f14c7a1a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:46.154573 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a93a7c-5944-4d08-af23-6ea4f14c7a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:46.154847 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a93a7c-5944-4d08-af23-6ea4f14c7a1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:46.155697 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a93a7c-5944-4d08-af23-6ea4f14c7a1a None admin] RBAC: Adding query filter params (role.id=85dcd5509fca4911881d45baf5f1d5a5, scope.project.id=c9f5c8ab86d44f42af60e2d8442a4f3d, user.id=a75fbca2c9d442b2bca8554b27dc944a) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:46.156010 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a93a7c-5944-4d08-af23-6ea4f14c7a1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:46.157752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a93a7c-5944-4d08-af23-6ea4f14c7a1a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:46.164795 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 136/269] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:50:45 2022] GET /identity/v3/role_assignments?user.id=a75fbca2c9d442b2bca8554b27dc944a&scope.project.id=c9f5c8ab86d44f42af60e2d8442a4f3d&role.id=85dcd5509fca4911881d45baf5f1d5a5 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:46.281013 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d22be4-a1e6-44cc-86eb-977ccf03a6c4 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:46.281716 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d22be4-a1e6-44cc-86eb-977ccf03a6c4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:46.282195 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d22be4-a1e6-44cc-86eb-977ccf03a6c4 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:46.284711 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 134/270] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:46.290986 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d5095-f23d-4bd9-ae9f-1f6778b15636 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:46.291639 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d5095-f23d-4bd9-ae9f-1f6778b15636 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:46.292141 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d5095-f23d-4bd9-ae9f-1f6778b15636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:46.318701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-802d5095-f23d-4bd9-ae9f-1f6778b15636 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:46.320387 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-802d5095-f23d-4bd9-ae9f-1f6778b15636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:46.425051 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 137/271] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 15:50:46.450554 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d22be4-a1e6-44cc-86eb-977ccf03a6c4 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:46.551765 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a1316-7bd8-4f91-ae87-670727fadd2f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:46.552071 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a1316-7bd8-4f91-ae87-670727fadd2f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:46.552333 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a1316-7bd8-4f91-ae87-670727fadd2f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:46.553804 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 138/272] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:46.559782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31455c2e-ea8a-4d2b-8855-7f92cca8e384 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:46.560391 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31455c2e-ea8a-4d2b-8855-7f92cca8e384 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:46.560893 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31455c2e-ea8a-4d2b-8855-7f92cca8e384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:46.583431 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-31455c2e-ea8a-4d2b-8855-7f92cca8e384 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:46.584553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-31455c2e-ea8a-4d2b-8855-7f92cca8e384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:46.593504 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d22be4-a1e6-44cc-86eb-977ccf03a6c4 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:46.633547 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:46.641328 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a483f5fb-b687-44a0-908e-6acdf2eb1531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:46.642163 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483f5fb-b687-44a0-908e-6acdf2eb1531 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:46.642450 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483f5fb-b687-44a0-908e-6acdf2eb1531 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:46.642717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483f5fb-b687-44a0-908e-6acdf2eb1531 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:46.655715 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a483f5fb-b687-44a0-908e-6acdf2eb1531 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:46.664647 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-a483f5fb-b687-44a0-908e-6acdf2eb1531 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:50:46.664647 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:46.664647 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:46.664647 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:46.664647 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:46.664647 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:46.664647 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:46.664647 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:46.664647 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:46.664647 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:46.664647 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:46.664647 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:46.664647 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:46.664647 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:46.664647 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:46.664647 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:46.665697 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:46.665697 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:46.665697 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:46.665697 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:46.665697 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:46.665697 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:46.665697 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:46.665697 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:46.665697 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:46.665697 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:46.665697 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:46.665697 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:46.665697 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:46.665697 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:46.665697 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:46.665697 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:46.666977 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:46.666977 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:46.666977 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:46.666977 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:46.666977 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:46.666977 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:46.666977 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:46.666977 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:46.666977 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:46.666977 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:46.666977 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:50:46.666977 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:50:46.666977 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 135/273] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:50:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:46.671950 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a483f5fb-b687-44a0-908e-6acdf2eb1531 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:46.678663 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:46.692080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 139/274] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 15:50:46.718026 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31455c2e-ea8a-4d2b-8855-7f92cca8e384 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:46.819599 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a483f5fb-b687-44a0-908e-6acdf2eb1531 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:46.850117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31455c2e-ea8a-4d2b-8855-7f92cca8e384 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:46.867685 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf64c3-b151-4c97-9317-db77b4cdc545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:46.868361 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf64c3-b151-4c97-9317-db77b4cdc545 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:46.868645 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf64c3-b151-4c97-9317-db77b4cdc545 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:46.868979 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf64c3-b151-4c97-9317-db77b4cdc545 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:46.870094 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf64c3-b151-4c97-9317-db77b4cdc545 None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:46.870409 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf64c3-b151-4c97-9317-db77b4cdc545 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:46.872239 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf64c3-b151-4c97-9317-db77b4cdc545 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:46.883823 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 136/275] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 15:50:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:50:46.890701 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf64c3-b151-4c97-9317-db77b4cdc545 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:46.898075 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50cd0dd-4400-49ec-a0c1-b330c0d0b897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:46.899232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50cd0dd-4400-49ec-a0c1-b330c0d0b897 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:46.899721 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50cd0dd-4400-49ec-a0c1-b330c0d0b897 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:46.900246 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50cd0dd-4400-49ec-a0c1-b330c0d0b897 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:46.901817 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50cd0dd-4400-49ec-a0c1-b330c0d0b897 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:46.903674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50cd0dd-4400-49ec-a0c1-b330c0d0b897 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:46.906092 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 140/276] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:50:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:50:46.912289 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50cd0dd-4400-49ec-a0c1-b330c0d0b897 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:46.972584 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:46.984010 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:47.027914 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50cd0dd-4400-49ec-a0c1-b330c0d0b897 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:47.030045 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf64c3-b151-4c97-9317-db77b4cdc545 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:47.065789 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83672c1-29f1-44f1-b124-e5bd9e20473d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:47.067216 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83672c1-29f1-44f1-b124-e5bd9e20473d None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:47.067680 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83672c1-29f1-44f1-b124-e5bd9e20473d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:47.068143 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83672c1-29f1-44f1-b124-e5bd9e20473d None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:47.068984 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b424ec-85cf-43c4-8347-958437704b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:47.069348 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83672c1-29f1-44f1-b124-e5bd9e20473d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:47.070033 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b424ec-85cf-43c4-8347-958437704b3f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:47.070381 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b424ec-85cf-43c4-8347-958437704b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:47.070723 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b424ec-85cf-43c4-8347-958437704b3f None admin] PATH_INFO: `/v3/users/872c16e0825d4cd7b281cba316b4579b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:47.071764 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83672c1-29f1-44f1-b124-e5bd9e20473d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:47.088050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b424ec-85cf-43c4-8347-958437704b3f None admin] RBAC: Authorizing `identity:get_user(user_id=872c16e0825d4cd7b281cba316b4579b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:47.089784 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b424ec-85cf-43c4-8347-958437704b3f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:47.095555 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 137/277] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:46 2022] GET /identity/v3/users/872c16e0825d4cd7b281cba316b4579b => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:47.100284 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b424ec-85cf-43c4-8347-958437704b3f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:47.105316 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 141/278] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 15:50:46 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 15:50:47.250423 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b424ec-85cf-43c4-8347-958437704b3f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:47.264679 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:47.294796 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f280809-8dd7-43dd-811f-8577a359f0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:47.295530 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f280809-8dd7-43dd-811f-8577a359f0c3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:47.295820 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f280809-8dd7-43dd-811f-8577a359f0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:47.296106 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f280809-8dd7-43dd-811f-8577a359f0c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:47.297068 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f280809-8dd7-43dd-811f-8577a359f0c3 None admin] RBAC: Adding query filter params (role.id=54195e660fc349978b39d5794765a095, user.id=872c16e0825d4cd7b281cba316b4579b, scope.system=all) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:47.297392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f280809-8dd7-43dd-811f-8577a359f0c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:47.298959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f280809-8dd7-43dd-811f-8577a359f0c3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:47.308769 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 138/279] 10.222.0.52 () {56 vars in 1308 bytes} [Thu Jul 21 15:50:47 2022] GET /identity/v3/role_assignments?user.id=872c16e0825d4cd7b281cba316b4579b&scope.system=all&role.id=54195e660fc349978b39d5794765a095 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:47.370014 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e9235-5e46-4a71-b634-1e7106d0ed89 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:47.370641 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e9235-5e46-4a71-b634-1e7106d0ed89 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:47.371129 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e9235-5e46-4a71-b634-1e7106d0ed89 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:47.373188 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 142/280] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:47.379894 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732a718-7e40-43ab-832e-353c201ae68f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:47.380231 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732a718-7e40-43ab-832e-353c201ae68f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:47.380504 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732a718-7e40-43ab-832e-353c201ae68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:47.402640 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0732a718-7e40-43ab-832e-353c201ae68f None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:47.403932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0732a718-7e40-43ab-832e-353c201ae68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:47.500549 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 139/281] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:47.523310 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396e9235-5e46-4a71-b634-1e7106d0ed89 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:47.648167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396e9235-5e46-4a71-b634-1e7106d0ed89 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:47.667870 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:47.687795 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e6b7e-4011-4335-8a37-2fd27103ee3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:47.688836 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e6b7e-4011-4335-8a37-2fd27103ee3b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:47.689288 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e6b7e-4011-4335-8a37-2fd27103ee3b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:47.689749 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e6b7e-4011-4335-8a37-2fd27103ee3b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:47.709832 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933e6b7e-4011-4335-8a37-2fd27103ee3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:47.718164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-933e6b7e-4011-4335-8a37-2fd27103ee3b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 15:50:47.718164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:47.718164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:47.718164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:47.718164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:47.718164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:47.718164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:47.718164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:47.718164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:47.718164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:47.718164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:47.718164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:47.718164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:47.718164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:47.718164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:47.718164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:47.719128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:47.719128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:47.719128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:47.719128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:47.719128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:47.719128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:47.719128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:47.719128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:47.719128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:47.719128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:47.719128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:47.719128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:47.719128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:47.719128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:47.719128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:47.719128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:47.719960 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:47.719960 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:47.719960 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:47.719960 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:47.719960 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:47.719960 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:47.719960 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:47.719960 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:47.719960 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:47.719960 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:47.719960 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 15:50:47.719960 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:50:47.719960 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 143/282] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:50:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:47.730798 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732a718-7e40-43ab-832e-353c201ae68f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:47.740951 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69651f1-3a1d-44fd-8f8a-66f85c30a283 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:47.741244 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69651f1-3a1d-44fd-8f8a-66f85c30a283 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:47.741515 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69651f1-3a1d-44fd-8f8a-66f85c30a283 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:47.742980 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 144/283] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:47.751236 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7f4e9-9957-41f9-a79d-fa78c764f993 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:47.751833 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7f4e9-9957-41f9-a79d-fa78c764f993 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:47.752308 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7f4e9-9957-41f9-a79d-fa78c764f993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:47.773167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e5c7f4e9-9957-41f9-a79d-fa78c764f993 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:47.774875 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e5c7f4e9-9957-41f9-a79d-fa78c764f993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:47.892780 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 145/284] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 15:50:47.926200 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732a718-7e40-43ab-832e-353c201ae68f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:47.928625 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:47.929036 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7f4e9-9957-41f9-a79d-fa78c764f993 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:47.988295 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de26c0ac-ae19-4453-baf0-8a95d8c13f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:47.988970 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26c0ac-ae19-4453-baf0-8a95d8c13f4c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:47.989250 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26c0ac-ae19-4453-baf0-8a95d8c13f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:47.989521 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26c0ac-ae19-4453-baf0-8a95d8c13f4c None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:47.990307 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de26c0ac-ae19-4453-baf0-8a95d8c13f4c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:47.990615 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de26c0ac-ae19-4453-baf0-8a95d8c13f4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:47.992022 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de26c0ac-ae19-4453-baf0-8a95d8c13f4c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:48.010891 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 140/285] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 15:50:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:48.017134 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de26c0ac-ae19-4453-baf0-8a95d8c13f4c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:48.137190 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7f4e9-9957-41f9-a79d-fa78c764f993 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:48.154171 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:48.196182 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc3817-84c0-44a2-bd46-e20d99244ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:48.197056 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc3817-84c0-44a2-bd46-e20d99244ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:48.197400 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc3817-84c0-44a2-bd46-e20d99244ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:48.197775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc3817-84c0-44a2-bd46-e20d99244ca1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:48.223710 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de26c0ac-ae19-4453-baf0-8a95d8c13f4c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:48.226054 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fc3817-84c0-44a2-bd46-e20d99244ca1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:48.249267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-f8fc3817-84c0-44a2-bd46-e20d99244ca1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:50:48.249267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:48.249267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:48.249267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:48.249267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:48.249267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:48.249267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:48.249267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:48.249267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:48.249267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:48.249267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:48.249267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:48.249267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:48.249267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:48.249267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:48.249267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:48.250457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:48.250457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:48.250457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:48.250457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:48.250457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:48.250457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:48.250457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:48.250457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:48.250457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:48.250457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:48.250457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:48.250457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:48.250457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:48.250457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:48.250457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:48.250457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:48.251475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:48.251475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:48.251475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:48.251475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:48.251475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:48.251475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:48.251475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:48.251475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:48.251475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:48.251475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:48.251475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:50:48.251475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:50:48.254401 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 146/286] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:50:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:48.263299 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc3817-84c0-44a2-bd46-e20d99244ca1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:48.319012 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f08080a-cce8-4eec-962d-366daf10bf3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:48.319923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08080a-cce8-4eec-962d-366daf10bf3a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:48.320212 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08080a-cce8-4eec-962d-366daf10bf3a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:48.320494 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08080a-cce8-4eec-962d-366daf10bf3a None admin] PATH_INFO: `/v3/users/1e906c799a234414953cdb2cf27362a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:48.321834 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f08080a-cce8-4eec-962d-366daf10bf3a None admin] RBAC: Authorizing `identity:get_user(user_id=1e906c799a234414953cdb2cf27362a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:48.323363 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f08080a-cce8-4eec-962d-366daf10bf3a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:48.325239 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 141/287] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:48 2022] GET /identity/v3/users/1e906c799a234414953cdb2cf27362a1 => generated 297 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:48.335227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f08080a-cce8-4eec-962d-366daf10bf3a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:48.387263 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc3817-84c0-44a2-bd46-e20d99244ca1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:48.430593 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdcf6b1-42ee-4081-82d8-9f1d74a29f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:48.431678 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcf6b1-42ee-4081-82d8-9f1d74a29f64 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:48.432173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcf6b1-42ee-4081-82d8-9f1d74a29f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:48.432724 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcf6b1-42ee-4081-82d8-9f1d74a29f64 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:48.433905 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdcf6b1-42ee-4081-82d8-9f1d74a29f64 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:48.434447 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdcf6b1-42ee-4081-82d8-9f1d74a29f64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:48.436325 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdcf6b1-42ee-4081-82d8-9f1d74a29f64 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:48.447158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 147/288] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 15:50:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:48.452874 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdcf6b1-42ee-4081-82d8-9f1d74a29f64 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:48.453810 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:48.498049 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:48.572050 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdcf6b1-42ee-4081-82d8-9f1d74a29f64 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:48.616516 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddca981-db9e-4cce-8e5f-a0e71d1d7978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:48.618038 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddca981-db9e-4cce-8e5f-a0e71d1d7978 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:48.618557 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddca981-db9e-4cce-8e5f-a0e71d1d7978 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:48.619030 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddca981-db9e-4cce-8e5f-a0e71d1d7978 None admin] PATH_INFO: `/v3/users/19a547136f6d40e097f7fbe83286fc89` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:48.620743 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddca981-db9e-4cce-8e5f-a0e71d1d7978 None admin] RBAC: Authorizing `identity:get_user(user_id=19a547136f6d40e097f7fbe83286fc89)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:48.622751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddca981-db9e-4cce-8e5f-a0e71d1d7978 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:48.624922 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 148/289] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:48 2022] GET /identity/v3/users/19a547136f6d40e097f7fbe83286fc89 => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:50:48.631876 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddca981-db9e-4cce-8e5f-a0e71d1d7978 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:48.705601 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f08080a-cce8-4eec-962d-366daf10bf3a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:48.746971 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddca981-db9e-4cce-8e5f-a0e71d1d7978 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:48.758040 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:48.788614 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b658d-fd46-4be2-a4ec-cd6a2797ad3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:48.789723 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b658d-fd46-4be2-a4ec-cd6a2797ad3c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:48.790259 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b658d-fd46-4be2-a4ec-cd6a2797ad3c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:48.790720 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b658d-fd46-4be2-a4ec-cd6a2797ad3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:48.849261 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9463f22-8970-4ee9-a645-bbfb1c61ec7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:48.849261 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9463f22-8970-4ee9-a645-bbfb1c61ec7c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:48.849261 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9463f22-8970-4ee9-a645-bbfb1c61ec7c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:48.849261 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9463f22-8970-4ee9-a645-bbfb1c61ec7c None admin] PATH_INFO: `/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:48.854403 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9463f22-8970-4ee9-a645-bbfb1c61ec7c None admin] RBAC: Authorizing `identity:get_project(project_id=c9f5c8ab86d44f42af60e2d8442a4f3d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:48.856253 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9463f22-8970-4ee9-a645-bbfb1c61ec7c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:48.857733 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 142/290] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:50:48 2022] GET /identity/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d => generated 302 bytes in 524 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:48.874604 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9463f22-8970-4ee9-a645-bbfb1c61ec7c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:48.891006 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b658d-fd46-4be2-a4ec-cd6a2797ad3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:48.892649 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b658d-fd46-4be2-a4ec-cd6a2797ad3c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:49.062047 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:49.066531 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9463f22-8970-4ee9-a645-bbfb1c61ec7c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:49.069362 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 149/291] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:50:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 15:50:49.076541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b658d-fd46-4be2-a4ec-cd6a2797ad3c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:49.081553 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:49.141416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac3d2e-9697-4cf3-9f89-5a9fb28e633e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:49.142364 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac3d2e-9697-4cf3-9f89-5a9fb28e633e None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:49.142637 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac3d2e-9697-4cf3-9f89-5a9fb28e633e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:49.143429 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac3d2e-9697-4cf3-9f89-5a9fb28e633e None admin] PATH_INFO: `/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d/users/1e906c799a234414953cdb2cf27362a1/roles/f4d0b371e4b140d9a9f6d9c6983c0ec8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:49.148545 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac3d2e-9697-4cf3-9f89-5a9fb28e633e None admin] RBAC: Authorizing `identity:create_grant(project_id=c9f5c8ab86d44f42af60e2d8442a4f3d, user_id=1e906c799a234414953cdb2cf27362a1, role_id=f4d0b371e4b140d9a9f6d9c6983c0ec8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:49.150062 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac3d2e-9697-4cf3-9f89-5a9fb28e633e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:49.173146 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 143/292] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 15:50:48 2022] PUT /identity/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d/users/1e906c799a234414953cdb2cf27362a1/roles/f4d0b371e4b140d9a9f6d9c6983c0ec8 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 15:50:49.247043 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b658d-fd46-4be2-a4ec-cd6a2797ad3c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:49.286228 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed4d3e1-404d-44bc-9ba1-6f21ce7f1c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:49.287145 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4d3e1-404d-44bc-9ba1-6f21ce7f1c99 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:49.287428 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4d3e1-404d-44bc-9ba1-6f21ce7f1c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:49.287701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4d3e1-404d-44bc-9ba1-6f21ce7f1c99 None admin] PATH_INFO: `/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:49.288977 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed4d3e1-404d-44bc-9ba1-6f21ce7f1c99 None admin] RBAC: Authorizing `identity:get_project(project_id=c9f5c8ab86d44f42af60e2d8442a4f3d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:49.290520 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed4d3e1-404d-44bc-9ba1-6f21ce7f1c99 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:49.292749 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 150/293] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:50:49 2022] GET /identity/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:49.298507 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac3d2e-9697-4cf3-9f89-5a9fb28e633e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:49.441709 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac3d2e-9697-4cf3-9f89-5a9fb28e633e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:49.480549 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a389a9e-4d4a-40b0-a4eb-2565abe0d0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:49.481222 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a389a9e-4d4a-40b0-a4eb-2565abe0d0a3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:49.481512 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a389a9e-4d4a-40b0-a4eb-2565abe0d0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:49.481823 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a389a9e-4d4a-40b0-a4eb-2565abe0d0a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:49.482742 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a389a9e-4d4a-40b0-a4eb-2565abe0d0a3 None admin] RBAC: Adding query filter params (role.id=54195e660fc349978b39d5794765a095, scope.project.id=c9f5c8ab86d44f42af60e2d8442a4f3d, user.id=19a547136f6d40e097f7fbe83286fc89) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:49.483112 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a389a9e-4d4a-40b0-a4eb-2565abe0d0a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:49.484581 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a389a9e-4d4a-40b0-a4eb-2565abe0d0a3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:49.491723 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 144/294] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:50:49 2022] GET /identity/v3/role_assignments?user.id=19a547136f6d40e097f7fbe83286fc89&scope.project.id=c9f5c8ab86d44f42af60e2d8442a4f3d&role.id=54195e660fc349978b39d5794765a095 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:49.833184 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37362cba-5a6e-4c3c-b59a-9d373d72f12c None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:49.833541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37362cba-5a6e-4c3c-b59a-9d373d72f12c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:49.833916 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37362cba-5a6e-4c3c-b59a-9d373d72f12c None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:49.835539 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 151/295] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:49.836973 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858c64b-294d-467f-bd64-829caa8cb03a None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:49.837263 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858c64b-294d-467f-bd64-829caa8cb03a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:49.837541 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858c64b-294d-467f-bd64-829caa8cb03a None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:49.838954 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 145/296] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:49.841213 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac3992-4664-467e-a49e-4b6e210718ea None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:49.841501 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac3992-4664-467e-a49e-4b6e210718ea None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:49.841814 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac3992-4664-467e-a49e-4b6e210718ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:49.844349 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57617d3e-0c16-4bd9-b950-54b62fec9a1b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:49.844641 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57617d3e-0c16-4bd9-b950-54b62fec9a1b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:49.844924 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57617d3e-0c16-4bd9-b950-54b62fec9a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:49.860440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-85ac3992-4664-467e-a49e-4b6e210718ea None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:49.861706 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-85ac3992-4664-467e-a49e-4b6e210718ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:49.864785 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-57617d3e-0c16-4bd9-b950-54b62fec9a1b None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:49.866015 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-57617d3e-0c16-4bd9-b950-54b62fec9a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:49.888879 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:49.941370 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:49.950816 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 152/297] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:49.957636 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c3d41-f72e-42cd-8e9a-210412fc2e15 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:49.957636 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c3d41-f72e-42cd-8e9a-210412fc2e15 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:49.957636 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c3d41-f72e-42cd-8e9a-210412fc2e15 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:49.957636 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 153/298] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:49.960385 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 146/299] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:49.966906 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd94251-642d-47b0-8c42-1a3b6373b6a2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:49.967203 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd94251-642d-47b0-8c42-1a3b6373b6a2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:49.967490 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd94251-642d-47b0-8c42-1a3b6373b6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:49.975959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57617d3e-0c16-4bd9-b950-54b62fec9a1b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:49.990704 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3bd94251-642d-47b0-8c42-1a3b6373b6a2 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:49.991903 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3bd94251-642d-47b0-8c42-1a3b6373b6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:50.081371 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 154/300] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:50.082489 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd94251-642d-47b0-8c42-1a3b6373b6a2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:50.109792 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57617d3e-0c16-4bd9-b950-54b62fec9a1b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:50.149613 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff4c3de-7585-45d4-a201-efb91efda280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:50.150862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4c3de-7585-45d4-a201-efb91efda280 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:50.151295 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4c3de-7585-45d4-a201-efb91efda280 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:50.151717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4c3de-7585-45d4-a201-efb91efda280 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:50.164039 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff4c3de-7585-45d4-a201-efb91efda280 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:50.172687 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-8ff4c3de-7585-45d4-a201-efb91efda280 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:50:50.172687 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:50.172687 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:50.172687 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:50.172687 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:50.172687 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:50.172687 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:50.172687 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:50.172687 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:50.172687 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:50.172687 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:50.172687 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:50.172687 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:50.172687 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:50.172687 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:50.172687 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:50.173850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:50.173850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:50.173850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:50.173850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:50.173850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:50.173850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:50.173850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:50.173850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:50.173850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:50.173850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:50.173850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:50.173850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:50.173850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:50.173850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:50.173850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:50.173850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:50.174893 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:50.174893 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:50.174893 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:50.174893 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:50.174893 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:50.174893 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:50.174893 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:50.174893 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:50.174893 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:50.174893 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:50.174893 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:50:50.174893 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:50:50.177084 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 147/301] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:50:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:50:50.178195 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff4c3de-7585-45d4-a201-efb91efda280 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:50.198707 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:50.206310 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd94251-642d-47b0-8c42-1a3b6373b6a2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:50.235291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:50.248512 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca2e13-2ceb-4bd2-8c4f-341e10732666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:50.250558 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca2e13-2ceb-4bd2-8c4f-341e10732666 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:50.250989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca2e13-2ceb-4bd2-8c4f-341e10732666 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:50.251413 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca2e13-2ceb-4bd2-8c4f-341e10732666 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:50.253366 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ca2e13-2ceb-4bd2-8c4f-341e10732666 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:50.255077 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ca2e13-2ceb-4bd2-8c4f-341e10732666 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:50.257011 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 155/302] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:50:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:50.257996 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca2e13-2ceb-4bd2-8c4f-341e10732666 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:50.314365 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff4c3de-7585-45d4-a201-efb91efda280 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:50.351859 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7584b1a9-316c-45a9-83dc-4920cfd4152a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:50.352688 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584b1a9-316c-45a9-83dc-4920cfd4152a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:50.352968 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584b1a9-316c-45a9-83dc-4920cfd4152a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:50.353242 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584b1a9-316c-45a9-83dc-4920cfd4152a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:50.364775 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7584b1a9-316c-45a9-83dc-4920cfd4152a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:50.370174 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca2e13-2ceb-4bd2-8c4f-341e10732666 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:50.372656 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-7584b1a9-316c-45a9-83dc-4920cfd4152a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 15:50:50.372656 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:50.372656 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:50.372656 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:50.372656 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:50.372656 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:50.372656 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:50.372656 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:50.372656 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:50.372656 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:50.372656 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:50.372656 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:50.372656 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:50.372656 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:50.372656 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:50.372656 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:50.373598 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:50.373598 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:50.373598 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:50.373598 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:50.373598 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:50.373598 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:50.373598 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:50.373598 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:50.373598 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:50.373598 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:50.373598 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:50.373598 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:50.373598 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:50.373598 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:50.373598 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:50.373598 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:50.374583 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:50.374583 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:50.374583 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:50.374583 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:50.374583 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:50.374583 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:50.374583 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:50.374583 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:50.374583 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:50.374583 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:50.374583 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 15:50:50.374583 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:50:50.379112 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 148/303] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:50:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:50.380041 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7584b1a9-316c-45a9-83dc-4920cfd4152a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:50.412259 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719056f1-67ae-484c-a85a-9cda75eaa89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:50.413164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719056f1-67ae-484c-a85a-9cda75eaa89b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:50.413372 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719056f1-67ae-484c-a85a-9cda75eaa89b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:50.413688 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719056f1-67ae-484c-a85a-9cda75eaa89b None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:50.414459 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719056f1-67ae-484c-a85a-9cda75eaa89b None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:50.414867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719056f1-67ae-484c-a85a-9cda75eaa89b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:50.416551 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719056f1-67ae-484c-a85a-9cda75eaa89b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:50.429933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 156/304] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 15:50:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:50.430881 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719056f1-67ae-484c-a85a-9cda75eaa89b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:50.474991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:50.503015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:50.505611 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7584b1a9-316c-45a9-83dc-4920cfd4152a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:50.542647 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f446f1b-5e13-4aa0-9565-4e96db8ba024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:50.543319 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f446f1b-5e13-4aa0-9565-4e96db8ba024 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:50.543599 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f446f1b-5e13-4aa0-9565-4e96db8ba024 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:50.543874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f446f1b-5e13-4aa0-9565-4e96db8ba024 None admin] PATH_INFO: `/v3/groups` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:50.544567 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f446f1b-5e13-4aa0-9565-4e96db8ba024 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:50.546057 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f446f1b-5e13-4aa0-9565-4e96db8ba024 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:50.562570 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719056f1-67ae-484c-a85a-9cda75eaa89b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:50.567324 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 149/305] 10.222.0.52 () {60 vars in 1134 bytes} [Thu Jul 21 15:50:50 2022] POST /identity/v3/groups => generated 229 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:50:50.568220 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f446f1b-5e13-4aa0-9565-4e96db8ba024 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:50.620102 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e10621-f257-42f1-b3e3-d15251af66de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:50.620707 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e10621-f257-42f1-b3e3-d15251af66de None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:50.620986 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e10621-f257-42f1-b3e3-d15251af66de None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:50.621321 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e10621-f257-42f1-b3e3-d15251af66de None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:50.622146 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e10621-f257-42f1-b3e3-d15251af66de None admin] RBAC: Adding query filter params (name=reader) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:50.622458 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e10621-f257-42f1-b3e3-d15251af66de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:50.623881 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e10621-f257-42f1-b3e3-d15251af66de None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:50.637445 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 157/306] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 15:50:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:50.644520 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e10621-f257-42f1-b3e3-d15251af66de None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:50.744377 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f446f1b-5e13-4aa0-9565-4e96db8ba024 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:50.770440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e10621-f257-42f1-b3e3-d15251af66de None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:50.795071 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28fa77a-ddd9-428d-b710-352d2518f2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:50.795647 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:50.797409 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28fa77a-ddd9-428d-b710-352d2518f2fd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:50.797899 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28fa77a-ddd9-428d-b710-352d2518f2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:50.798416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28fa77a-ddd9-428d-b710-352d2518f2fd None admin] PATH_INFO: `/v3/users/872c16e0825d4cd7b281cba316b4579b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:50.806060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28fa77a-ddd9-428d-b710-352d2518f2fd None admin] RBAC: Authorizing `identity:get_user(user_id=872c16e0825d4cd7b281cba316b4579b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:50.808088 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28fa77a-ddd9-428d-b710-352d2518f2fd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:50.809725 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 150/307] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:50 2022] GET /identity/v3/users/872c16e0825d4cd7b281cba316b4579b => generated 293 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:50.817234 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28fa77a-ddd9-428d-b710-352d2518f2fd None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:50.822571 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:50.823806 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d7357-f128-48e3-80ed-6f3bf650db39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:50.824591 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d7357-f128-48e3-80ed-6f3bf650db39 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:50.824929 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d7357-f128-48e3-80ed-6f3bf650db39 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:50.825235 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d7357-f128-48e3-80ed-6f3bf650db39 None admin] PATH_INFO: `/v3/users/71fd9dac8a29493bb9b393d3e2680e38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:50.852737 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53d7357-f128-48e3-80ed-6f3bf650db39 None admin] RBAC: Authorizing `identity:get_user(user_id=71fd9dac8a29493bb9b393d3e2680e38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:50.857082 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53d7357-f128-48e3-80ed-6f3bf650db39 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:50.861895 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 158/308] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:50 2022] GET /identity/v3/users/71fd9dac8a29493bb9b393d3e2680e38 => generated 293 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:50:50.868218 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d7357-f128-48e3-80ed-6f3bf650db39 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:50.970025 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28fa77a-ddd9-428d-b710-352d2518f2fd None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:50.999576 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d7357-f128-48e3-80ed-6f3bf650db39 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:51.012045 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327e19f-111a-4900-8ec0-052ed18cc33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:51.013062 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327e19f-111a-4900-8ec0-052ed18cc33f None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:51.013423 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327e19f-111a-4900-8ec0-052ed18cc33f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:51.013904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327e19f-111a-4900-8ec0-052ed18cc33f None admin] PATH_INFO: `/v3/system/users/872c16e0825d4cd7b281cba316b4579b/roles/54195e660fc349978b39d5794765a095` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:51.015758 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327e19f-111a-4900-8ec0-052ed18cc33f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=872c16e0825d4cd7b281cba316b4579b, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:51.017350 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327e19f-111a-4900-8ec0-052ed18cc33f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:51.037713 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 151/309] 10.222.0.52 () {58 vars in 1334 bytes} [Thu Jul 21 15:50:50 2022] PUT /identity/v3/system/users/872c16e0825d4cd7b281cba316b4579b/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 15:50:51.049196 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d32be-cb4d-45fd-afd5-24d14273dd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:51.050945 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d32be-cb4d-45fd-afd5-24d14273dd42 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:51.051501 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d32be-cb4d-45fd-afd5-24d14273dd42 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:51.052024 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d32be-cb4d-45fd-afd5-24d14273dd42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:51.053940 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3d32be-cb4d-45fd-afd5-24d14273dd42 None admin] RBAC: Adding query filter params (role.id=f4d0b371e4b140d9a9f6d9c6983c0ec8, user.id=71fd9dac8a29493bb9b393d3e2680e38, scope.system=all) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:51.054380 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3d32be-cb4d-45fd-afd5-24d14273dd42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:51.056085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3d32be-cb4d-45fd-afd5-24d14273dd42 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:51.063203 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 159/310] 10.222.0.52 () {56 vars in 1308 bytes} [Thu Jul 21 15:50:50 2022] GET /identity/v3/role_assignments?user.id=71fd9dac8a29493bb9b393d3e2680e38&scope.system=all&role.id=f4d0b371e4b140d9a9f6d9c6983c0ec8 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:51.633789 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ea2ba-4c57-409a-a7ef-d5e757feb1f2 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:51.634278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ea2ba-4c57-409a-a7ef-d5e757feb1f2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:51.634278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ea2ba-4c57-409a-a7ef-d5e757feb1f2 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:51.635866 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 152/311] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:51.641648 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7201f-59b9-4d51-a4b4-5a6123ebc146 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:51.642147 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7201f-59b9-4d51-a4b4-5a6123ebc146 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:51.642224 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7201f-59b9-4d51-a4b4-5a6123ebc146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:51.663282 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-cde7201f-59b9-4d51-a4b4-5a6123ebc146 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:51.668046 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cde7201f-59b9-4d51-a4b4-5a6123ebc146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:51.705363 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:51.741422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 160/312] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:51.765073 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ea2ba-4c57-409a-a7ef-d5e757feb1f2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:51.839506 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:51.880392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ea2ba-4c57-409a-a7ef-d5e757feb1f2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:51.915946 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719a3d7-c426-406f-977b-3f4e27cce931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:51.916668 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719a3d7-c426-406f-977b-3f4e27cce931 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:51.916952 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719a3d7-c426-406f-977b-3f4e27cce931 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:51.917226 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719a3d7-c426-406f-977b-3f4e27cce931 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:51.930341 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f719a3d7-c426-406f-977b-3f4e27cce931 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:51.938526 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-f719a3d7-c426-406f-977b-3f4e27cce931 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 15:50:51.938526 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:51.938526 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:51.938526 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:51.938526 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:51.938526 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:51.938526 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:51.938526 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:51.938526 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:51.938526 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:51.938526 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:51.938526 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:51.938526 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:51.938526 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:51.938526 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:51.938526 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:51.940745 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:51.940745 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:51.940745 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:51.940745 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:51.940745 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:51.940745 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:51.940745 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:51.940745 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:51.940745 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:51.940745 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:51.940745 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:51.940745 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:51.940745 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:51.940745 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:51.940745 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:51.940745 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:51.942809 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:51.942809 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:51.942809 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:51.942809 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:51.942809 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:51.942809 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:51.942809 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:51.942809 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:51.942809 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:51.942809 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:51.942809 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 15:50:51.942809 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:50:51.946779 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 153/313] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:50:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:51.948769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde7201f-59b9-4d51-a4b4-5a6123ebc146 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:52.075257 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde7201f-59b9-4d51-a4b4-5a6123ebc146 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:52.111008 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c144b49-cc72-414f-9539-8785a059ae31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:52.112209 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c144b49-cc72-414f-9539-8785a059ae31 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:52.112489 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c144b49-cc72-414f-9539-8785a059ae31 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:52.112763 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c144b49-cc72-414f-9539-8785a059ae31 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:52.113524 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c144b49-cc72-414f-9539-8785a059ae31 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:52.113860 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c144b49-cc72-414f-9539-8785a059ae31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:52.115276 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c144b49-cc72-414f-9539-8785a059ae31 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:52.124480 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 161/314] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 15:50:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:52.133641 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719a3d7-c426-406f-977b-3f4e27cce931 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:52.257745 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719a3d7-c426-406f-977b-3f4e27cce931 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:52.287120 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:52.296307 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6b563-3d62-4e51-bcd3-e6743dea8f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:52.297198 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6b563-3d62-4e51-bcd3-e6743dea8f7b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:52.297914 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6b563-3d62-4e51-bcd3-e6743dea8f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:52.298204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6b563-3d62-4e51-bcd3-e6743dea8f7b None admin] PATH_INFO: `/v3/users/1e906c799a234414953cdb2cf27362a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:52.300159 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf6b563-3d62-4e51-bcd3-e6743dea8f7b None admin] RBAC: Authorizing `identity:get_user(user_id=1e906c799a234414953cdb2cf27362a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:52.301662 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf6b563-3d62-4e51-bcd3-e6743dea8f7b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:52.303459 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 154/315] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:52 2022] GET /identity/v3/users/1e906c799a234414953cdb2cf27362a1 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:52.309343 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c144b49-cc72-414f-9539-8785a059ae31 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:52.349546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:52.426051 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c144b49-cc72-414f-9539-8785a059ae31 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:52.461014 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c1869-0d3d-4231-a202-0eeebe20ab94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:52.461695 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c1869-0d3d-4231-a202-0eeebe20ab94 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:52.461979 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c1869-0d3d-4231-a202-0eeebe20ab94 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:52.462254 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c1869-0d3d-4231-a202-0eeebe20ab94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:52.535343 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c1869-0d3d-4231-a202-0eeebe20ab94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:52.536911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c1869-0d3d-4231-a202-0eeebe20ab94 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:52.571575 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:52.643959 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 162/316] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:50:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 15:50:52.651088 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6b563-3d62-4e51-bcd3-e6743dea8f7b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:52.770481 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6b563-3d62-4e51-bcd3-e6743dea8f7b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:52.814405 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d543b-af4f-4207-8dd1-e79248ba616d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:52.815699 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d543b-af4f-4207-8dd1-e79248ba616d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:52.816214 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d543b-af4f-4207-8dd1-e79248ba616d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:52.816665 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d543b-af4f-4207-8dd1-e79248ba616d None admin] PATH_INFO: `/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:52.818294 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6d543b-af4f-4207-8dd1-e79248ba616d None admin] RBAC: Authorizing `identity:get_project(project_id=c9f5c8ab86d44f42af60e2d8442a4f3d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:52.819787 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6d543b-af4f-4207-8dd1-e79248ba616d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:52.825727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 155/317] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:50:52 2022] GET /identity/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:52.830174 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c1869-0d3d-4231-a202-0eeebe20ab94 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:52.941608 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c1869-0d3d-4231-a202-0eeebe20ab94 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:52.985935 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:52.990103 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb5523-3900-46e3-b002-08be4c494804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:52.990774 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb5523-3900-46e3-b002-08be4c494804 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:52.991117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb5523-3900-46e3-b002-08be4c494804 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:52.991365 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb5523-3900-46e3-b002-08be4c494804 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:52.992255 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb5523-3900-46e3-b002-08be4c494804 None admin] RBAC: Adding query filter params (role.id=f4d0b371e4b140d9a9f6d9c6983c0ec8, scope.project.id=c9f5c8ab86d44f42af60e2d8442a4f3d, user.id=1e906c799a234414953cdb2cf27362a1) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:52.992565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb5523-3900-46e3-b002-08be4c494804 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:52.994770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb5523-3900-46e3-b002-08be4c494804 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:53.001838 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 163/318] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:50:52 2022] GET /identity/v3/role_assignments?user.id=1e906c799a234414953cdb2cf27362a1&scope.project.id=c9f5c8ab86d44f42af60e2d8442a4f3d&role.id=f4d0b371e4b140d9a9f6d9c6983c0ec8 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:53.046240 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d63d4-237c-4ff5-bb40-a2545a358b0a None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:53.046546 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d63d4-237c-4ff5-bb40-a2545a358b0a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:53.046854 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d63d4-237c-4ff5-bb40-a2545a358b0a None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:53.048529 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 156/319] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:53.054983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d24ef-4658-4cd4-9854-71f8179a763c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:53.055321 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d24ef-4658-4cd4-9854-71f8179a763c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:53.055540 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d24ef-4658-4cd4-9854-71f8179a763c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:53.077678 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c22d24ef-4658-4cd4-9854-71f8179a763c None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:53.078846 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c22d24ef-4658-4cd4-9854-71f8179a763c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:53.157339 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 164/320] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 15:50:53.180372 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d63d4-237c-4ff5-bb40-a2545a358b0a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:53.256423 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:53.302775 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d63d4-237c-4ff5-bb40-a2545a358b0a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:53.344501 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc52e8-8c14-4097-affc-4c20858e6d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:53.345208 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc52e8-8c14-4097-affc-4c20858e6d81 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:53.345487 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc52e8-8c14-4097-affc-4c20858e6d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:53.345897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc52e8-8c14-4097-affc-4c20858e6d81 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:53.347113 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc52e8-8c14-4097-affc-4c20858e6d81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:53.348857 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc52e8-8c14-4097-affc-4c20858e6d81 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:53.350626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 157/321] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:50:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:53.352050 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766215b7-1723-4a15-8dc2-c0db03849cdd None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:53.352346 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766215b7-1723-4a15-8dc2-c0db03849cdd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:53.352623 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766215b7-1723-4a15-8dc2-c0db03849cdd None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:53.354158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 165/322] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 15:50:53.355815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc52e8-8c14-4097-affc-4c20858e6d81 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:53.359395 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ee4cf-e079-4e93-af4a-ab958613cf47 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:53.359764 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ee4cf-e079-4e93-af4a-ab958613cf47 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:53.359936 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ee4cf-e079-4e93-af4a-ab958613cf47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:53.378555 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7a4ee4cf-e079-4e93-af4a-ab958613cf47 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:53.379769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7a4ee4cf-e079-4e93-af4a-ab958613cf47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:53.472167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 166/323] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 15:50:53.474221 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9a670-ccc6-41a6-9014-cc189eb67999 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:53.474710 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9a670-ccc6-41a6-9014-cc189eb67999 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:53.475059 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc52e8-8c14-4097-affc-4c20858e6d81 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:53.475228 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9a670-ccc6-41a6-9014-cc189eb67999 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:53.477554 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 167/324] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:53.482863 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec090427-a280-4636-9373-44bc55680461 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:53.483148 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec090427-a280-4636-9373-44bc55680461 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:53.483423 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec090427-a280-4636-9373-44bc55680461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:53.500400 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ec090427-a280-4636-9373-44bc55680461 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:53.501604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ec090427-a280-4636-9373-44bc55680461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:53.514620 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:53.520832 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6271c91-2249-43c4-801c-a9eb4313ad83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:53.521512 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6271c91-2249-43c4-801c-a9eb4313ad83 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:53.521819 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6271c91-2249-43c4-801c-a9eb4313ad83 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:53.522096 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6271c91-2249-43c4-801c-a9eb4313ad83 None admin] PATH_INFO: `/v3/groups` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:53.522814 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6271c91-2249-43c4-801c-a9eb4313ad83 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:53.524280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6271c91-2249-43c4-801c-a9eb4313ad83 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:53.549551 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 158/325] 10.222.0.52 () {60 vars in 1134 bytes} [Thu Jul 21 15:50:53 2022] POST /identity/v3/groups => generated 226 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:50:53.551231 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6271c91-2249-43c4-801c-a9eb4313ad83 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:53.576022 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:53.599701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 168/326] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 15:50:53.622310 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec090427-a280-4636-9373-44bc55680461 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:53.679829 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6271c91-2249-43c4-801c-a9eb4313ad83 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:53.728111 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d257532-c465-4eb0-afa7-c13db6941317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:53.729315 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d257532-c465-4eb0-afa7-c13db6941317 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:53.729949 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d257532-c465-4eb0-afa7-c13db6941317 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:53.730429 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d257532-c465-4eb0-afa7-c13db6941317 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:53.748354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d257532-c465-4eb0-afa7-c13db6941317 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:53.756298 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec090427-a280-4636-9373-44bc55680461 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:53.758485 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-6d257532-c465-4eb0-afa7-c13db6941317 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:50:53.758485 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:53.758485 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:53.758485 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:53.758485 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:53.758485 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:53.758485 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:53.758485 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:53.758485 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:53.758485 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:53.758485 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:53.758485 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:53.758485 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:53.758485 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:53.758485 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:53.758485 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:53.759375 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:53.759375 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:53.759375 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:53.759375 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:53.759375 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:53.759375 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:53.759375 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:53.759375 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:53.759375 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:53.759375 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:53.759375 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:53.759375 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:53.759375 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:53.759375 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:53.759375 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:53.759375 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:53.760192 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:53.760192 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:53.760192 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:53.760192 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:53.760192 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:53.760192 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:53.760192 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:53.760192 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:53.760192 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:53.760192 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:53.760192 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:50:53.760192 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:50:53.760192 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 159/327] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:50:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:53.765246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d257532-c465-4eb0-afa7-c13db6941317 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:53.807785 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f68568-7311-42a9-9ffa-a7980af3a8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:53.808643 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f68568-7311-42a9-9ffa-a7980af3a8a0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:53.808977 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f68568-7311-42a9-9ffa-a7980af3a8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:53.809264 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f68568-7311-42a9-9ffa-a7980af3a8a0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:53.826657 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f68568-7311-42a9-9ffa-a7980af3a8a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:53.834360 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-12f68568-7311-42a9-9ffa-a7980af3a8a0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 15:50:53.834360 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:53.834360 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:53.834360 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:53.834360 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:53.834360 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:53.834360 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:53.834360 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:53.834360 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:53.834360 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:53.834360 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:53.834360 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:53.834360 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:53.834360 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:53.834360 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:53.834360 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:53.835369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:53.835369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:53.835369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:53.835369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:53.835369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:53.835369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:53.835369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:53.835369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:53.835369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:53.835369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:53.835369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:53.835369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:53.835369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:53.835369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:53.835369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:53.835369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:53.836228 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:53.836228 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:53.836228 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:53.836228 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:53.836228 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:53.836228 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:53.836228 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:53.836228 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:53.836228 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:53.836228 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:53.836228 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 15:50:53.836228 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:50:53.836228 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 169/328] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:50:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:50:53.836921 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:53.842158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f68568-7311-42a9-9ffa-a7980af3a8a0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:53.905055 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d257532-c465-4eb0-afa7-c13db6941317 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:53.912112 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:53.962101 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88de169-1c4d-4b01-b1b4-a884ef66f759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:53.962880 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88de169-1c4d-4b01-b1b4-a884ef66f759 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:53.963335 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88de169-1c4d-4b01-b1b4-a884ef66f759 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:53.963760 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88de169-1c4d-4b01-b1b4-a884ef66f759 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:53.964965 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88de169-1c4d-4b01-b1b4-a884ef66f759 None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:53.966290 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88de169-1c4d-4b01-b1b4-a884ef66f759 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:53.969362 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88de169-1c4d-4b01-b1b4-a884ef66f759 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:53.982810 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 160/329] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 15:50:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:50:53.988971 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88de169-1c4d-4b01-b1b4-a884ef66f759 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:54.002047 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f68568-7311-42a9-9ffa-a7980af3a8a0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:54.056273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075b00e-7475-43a0-93f8-30c7e1d65475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:54.057039 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075b00e-7475-43a0-93f8-30c7e1d65475 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:54.057670 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075b00e-7475-43a0-93f8-30c7e1d65475 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:54.058214 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075b00e-7475-43a0-93f8-30c7e1d65475 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:54.059416 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075b00e-7475-43a0-93f8-30c7e1d65475 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:54.060516 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075b00e-7475-43a0-93f8-30c7e1d65475 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:54.063475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075b00e-7475-43a0-93f8-30c7e1d65475 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:54.082038 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 170/330] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 15:50:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:54.088719 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075b00e-7475-43a0-93f8-30c7e1d65475 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:54.122937 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88de169-1c4d-4b01-b1b4-a884ef66f759 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:54.158079 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:54.166747 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe4ec3-aef7-458a-bcb2-0bf73286bf4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:54.167807 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe4ec3-aef7-458a-bcb2-0bf73286bf4e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:54.168095 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe4ec3-aef7-458a-bcb2-0bf73286bf4e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:54.168375 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe4ec3-aef7-458a-bcb2-0bf73286bf4e None admin] PATH_INFO: `/v3/users/872c16e0825d4cd7b281cba316b4579b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:54.169724 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abe4ec3-aef7-458a-bcb2-0bf73286bf4e None admin] RBAC: Authorizing `identity:get_user(user_id=872c16e0825d4cd7b281cba316b4579b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:54.171401 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abe4ec3-aef7-458a-bcb2-0bf73286bf4e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:54.173736 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 161/331] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:53 2022] GET /identity/v3/users/872c16e0825d4cd7b281cba316b4579b => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:54.180534 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe4ec3-aef7-458a-bcb2-0bf73286bf4e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:54.223850 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075b00e-7475-43a0-93f8-30c7e1d65475 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:54.237521 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:54.270180 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb4488-ee94-4678-90c1-f12eecf6cbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:54.271134 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb4488-ee94-4678-90c1-f12eecf6cbe3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:54.271499 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb4488-ee94-4678-90c1-f12eecf6cbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:54.271782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb4488-ee94-4678-90c1-f12eecf6cbe3 None admin] PATH_INFO: `/v3/users/71fd9dac8a29493bb9b393d3e2680e38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:54.273150 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb4488-ee94-4678-90c1-f12eecf6cbe3 None admin] RBAC: Authorizing `identity:get_user(user_id=71fd9dac8a29493bb9b393d3e2680e38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:54.274672 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb4488-ee94-4678-90c1-f12eecf6cbe3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:54.276763 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 171/332] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:50:54 2022] GET /identity/v3/users/71fd9dac8a29493bb9b393d3e2680e38 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:50:54.283005 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb4488-ee94-4678-90c1-f12eecf6cbe3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:54.323868 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe4ec3-aef7-458a-bcb2-0bf73286bf4e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:54.378952 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a3e95-0cd9-4853-a409-84f5299f28e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:54.379645 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a3e95-0cd9-4853-a409-84f5299f28e7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:54.379935 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a3e95-0cd9-4853-a409-84f5299f28e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:54.380221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a3e95-0cd9-4853-a409-84f5299f28e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:54.381144 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507a3e95-0cd9-4853-a409-84f5299f28e7 None admin] RBAC: Adding query filter params (role.id=54195e660fc349978b39d5794765a095, user.id=872c16e0825d4cd7b281cba316b4579b, scope.system=all) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:54.381471 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507a3e95-0cd9-4853-a409-84f5299f28e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:54.384082 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507a3e95-0cd9-4853-a409-84f5299f28e7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:54.399358 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 162/333] 10.222.0.52 () {56 vars in 1308 bytes} [Thu Jul 21 15:50:54 2022] GET /identity/v3/role_assignments?user.id=872c16e0825d4cd7b281cba316b4579b&scope.system=all&role.id=54195e660fc349978b39d5794765a095 => generated 510 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:54.412258 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb4488-ee94-4678-90c1-f12eecf6cbe3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:54.454203 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad57e87e-0d4f-438f-9619-bba61ae8cfaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:54.455686 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57e87e-0d4f-438f-9619-bba61ae8cfaa None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:54.456030 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57e87e-0d4f-438f-9619-bba61ae8cfaa None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:54.456359 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57e87e-0d4f-438f-9619-bba61ae8cfaa None admin] PATH_INFO: `/v3/system/users/71fd9dac8a29493bb9b393d3e2680e38/roles/f4d0b371e4b140d9a9f6d9c6983c0ec8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:54.458163 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad57e87e-0d4f-438f-9619-bba61ae8cfaa None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=71fd9dac8a29493bb9b393d3e2680e38, role_id=f4d0b371e4b140d9a9f6d9c6983c0ec8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:54.459663 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad57e87e-0d4f-438f-9619-bba61ae8cfaa None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:54.472468 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 172/334] 10.222.0.52 () {58 vars in 1334 bytes} [Thu Jul 21 15:50:54 2022] PUT /identity/v3/system/users/71fd9dac8a29493bb9b393d3e2680e38/roles/f4d0b371e4b140d9a9f6d9c6983c0ec8 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 15:50:56.603057 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81eb018-baba-44c6-8da1-e507c3ff1aac None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:56.603779 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81eb018-baba-44c6-8da1-e507c3ff1aac None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:56.603779 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81eb018-baba-44c6-8da1-e507c3ff1aac None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:56.605274 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 163/335] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:56.610993 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320ff2c-031c-47e5-81f6-759a1428dd8b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:56.611660 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320ff2c-031c-47e5-81f6-759a1428dd8b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:56.611660 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320ff2c-031c-47e5-81f6-759a1428dd8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:56.635321 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0320ff2c-031c-47e5-81f6-759a1428dd8b None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:56.636829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0320ff2c-031c-47e5-81f6-759a1428dd8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:56.686700 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf420a8-cb63-472e-b813-13c947df4008 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:56.687467 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:56.687950 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf420a8-cb63-472e-b813-13c947df4008 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:56.688657 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf420a8-cb63-472e-b813-13c947df4008 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:56.693695 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 164/336] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 15:50:56.699129 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf739a36-665c-4302-9f39-c10c84e1a274 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:56.699744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf739a36-665c-4302-9f39-c10c84e1a274 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:56.700236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf739a36-665c-4302-9f39-c10c84e1a274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:56.735484 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 173/337] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:56.736896 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597952c-bda1-4fc7-a132-5591f3140f41 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:56.737184 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597952c-bda1-4fc7-a132-5591f3140f41 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:56.737463 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597952c-bda1-4fc7-a132-5591f3140f41 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:56.737699 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-cf739a36-665c-4302-9f39-c10c84e1a274 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:56.738805 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 174/338] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:56.739381 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cf739a36-665c-4302-9f39-c10c84e1a274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:56.744491 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5c4b2-629b-4f94-b5f9-f1893dd892ea None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:56.744773 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5c4b2-629b-4f94-b5f9-f1893dd892ea None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:56.745047 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5c4b2-629b-4f94-b5f9-f1893dd892ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:56.772435 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a6a5c4b2-629b-4f94-b5f9-f1893dd892ea None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:56.773767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a6a5c4b2-629b-4f94-b5f9-f1893dd892ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:56.813373 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:56.852500 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 175/339] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 15:50:56.853377 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5c4b2-629b-4f94-b5f9-f1893dd892ea None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:56.884087 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 165/340] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:56.911691 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf739a36-665c-4302-9f39-c10c84e1a274 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:56.982998 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5c4b2-629b-4f94-b5f9-f1893dd892ea None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:56.984752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:57.035485 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f223608-6e2f-479e-90b1-005abce20f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:57.036309 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f223608-6e2f-479e-90b1-005abce20f34 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:57.036589 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f223608-6e2f-479e-90b1-005abce20f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:57.040518 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f223608-6e2f-479e-90b1-005abce20f34 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:57.066750 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f223608-6e2f-479e-90b1-005abce20f34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:57.075164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-2f223608-6e2f-479e-90b1-005abce20f34 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:50:57.075164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:57.075164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:57.075164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:57.075164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:57.075164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:57.075164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:57.075164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:57.075164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:57.075164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:57.075164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:57.075164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:57.075164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:57.075164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:57.075164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:57.075164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:57.076214 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:57.076214 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:57.076214 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:57.076214 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:57.076214 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:57.076214 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:57.076214 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:57.076214 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:57.076214 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:57.076214 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:57.076214 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:57.076214 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:57.076214 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:57.076214 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:57.076214 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:57.076214 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:57.077096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:57.077096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:57.077096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:57.077096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:57.077096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:57.077096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:57.077096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:57.077096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:57.077096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:57.077096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:57.077096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:50:57.077096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:50:57.082115 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 176/341] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:50:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:57.083069 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f223608-6e2f-479e-90b1-005abce20f34 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:57.084941 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf739a36-665c-4302-9f39-c10c84e1a274 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:57.134487 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dbe18e-7ad1-4801-bf1f-190dd5296b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:57.135193 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dbe18e-7ad1-4801-bf1f-190dd5296b97 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:57.135477 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dbe18e-7ad1-4801-bf1f-190dd5296b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:57.135754 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dbe18e-7ad1-4801-bf1f-190dd5296b97 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:57.148075 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dbe18e-7ad1-4801-bf1f-190dd5296b97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:57.156175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-10dbe18e-7ad1-4801-bf1f-190dd5296b97 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:50:57.156175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:57.156175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:57.156175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:57.156175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:57.156175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:57.156175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:57.156175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:57.156175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:57.156175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:57.156175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:57.156175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:57.156175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:57.156175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:57.156175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:57.156175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:57.157174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:57.157174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:57.157174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:57.157174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:57.157174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:57.157174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:57.157174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:57.157174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:57.157174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:57.157174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:57.157174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:57.157174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:57.157174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:57.157174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:57.157174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:57.157174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:57.158050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:57.158050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:57.158050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:57.158050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:57.158050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:57.158050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:57.158050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:57.158050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:57.158050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:57.158050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:57.158050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:50:57.158050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:50:57.161929 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 166/342] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:50:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:50:57.162922 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dbe18e-7ad1-4801-bf1f-190dd5296b97 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:57.197794 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:57.292146 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dbe18e-7ad1-4801-bf1f-190dd5296b97 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:57.295879 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f223608-6e2f-479e-90b1-005abce20f34 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:57.328966 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feaf727-c2b7-4e0d-bd1f-ebb7cb1b10ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:57.329673 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feaf727-c2b7-4e0d-bd1f-ebb7cb1b10ed None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:57.330000 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feaf727-c2b7-4e0d-bd1f-ebb7cb1b10ed None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:57.330286 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feaf727-c2b7-4e0d-bd1f-ebb7cb1b10ed None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:57.331060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feaf727-c2b7-4e0d-bd1f-ebb7cb1b10ed None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:57.331380 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feaf727-c2b7-4e0d-bd1f-ebb7cb1b10ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:57.332791 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feaf727-c2b7-4e0d-bd1f-ebb7cb1b10ed None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:57.342162 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 167/343] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 15:50:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:57.342838 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feaf727-c2b7-4e0d-bd1f-ebb7cb1b10ed None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:57.367393 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c790a-afd4-463a-bed9-644ce2f701a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:57.368379 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c790a-afd4-463a-bed9-644ce2f701a4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:57.368768 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c790a-afd4-463a-bed9-644ce2f701a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:57.369057 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c790a-afd4-463a-bed9-644ce2f701a4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:57.389829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c790a-afd4-463a-bed9-644ce2f701a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:57.399086 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-ee3c790a-afd4-463a-bed9-644ce2f701a4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 15:50:57.399086 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:57.399086 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:57.399086 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:57.399086 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:57.399086 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:57.399086 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:57.399086 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:57.399086 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:57.399086 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:57.399086 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:57.399086 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:57.399086 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:57.399086 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:57.399086 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:57.399086 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:57.400117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:57.400117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:57.400117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:57.400117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:57.400117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:57.400117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:57.400117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:57.400117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:57.400117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:57.400117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:57.400117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:57.400117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:57.400117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:57.400117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:57.400117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:57.400117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:57.401002 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:57.401002 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:57.401002 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:57.401002 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:57.401002 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:57.401002 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:57.401002 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:57.401002 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:57.401002 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:57.401002 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:57.401002 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 15:50:57.401002 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:50:57.401002 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 177/344] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 15:50:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:57.401890 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c790a-afd4-463a-bed9-644ce2f701a4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:57.432057 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:57.469523 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:57.483845 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feaf727-c2b7-4e0d-bd1f-ebb7cb1b10ed None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:57.529791 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c790a-afd4-463a-bed9-644ce2f701a4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:57.532149 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c61b2-9348-4dc1-8724-c7c4f0bf215c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:57.532900 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c61b2-9348-4dc1-8724-c7c4f0bf215c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:57.537813 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c61b2-9348-4dc1-8724-c7c4f0bf215c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:57.538139 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c61b2-9348-4dc1-8724-c7c4f0bf215c None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:57.538948 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c61b2-9348-4dc1-8724-c7c4f0bf215c None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:57.539264 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c61b2-9348-4dc1-8724-c7c4f0bf215c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:57.540683 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c61b2-9348-4dc1-8724-c7c4f0bf215c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:57.555665 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 168/345] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 15:50:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:57.556547 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c61b2-9348-4dc1-8724-c7c4f0bf215c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:57.575735 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b76ab6-41f7-4bc0-b824-4645ad3aecd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:57.576663 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b76ab6-41f7-4bc0-b824-4645ad3aecd0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:57.577056 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b76ab6-41f7-4bc0-b824-4645ad3aecd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:57.577440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b76ab6-41f7-4bc0-b824-4645ad3aecd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:57.731737 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b76ab6-41f7-4bc0-b824-4645ad3aecd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:57.734490 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b76ab6-41f7-4bc0-b824-4645ad3aecd0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:57.767468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c61b2-9348-4dc1-8724-c7c4f0bf215c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:57.781140 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:57.836574 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408a90a-46bd-4b24-b596-53ec90b641f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:57.837418 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408a90a-46bd-4b24-b596-53ec90b641f5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:57.837734 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408a90a-46bd-4b24-b596-53ec90b641f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:57.838095 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408a90a-46bd-4b24-b596-53ec90b641f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:57.839066 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6408a90a-46bd-4b24-b596-53ec90b641f5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:57.839554 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6408a90a-46bd-4b24-b596-53ec90b641f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:57.842151 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6408a90a-46bd-4b24-b596-53ec90b641f5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:57.865166 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 169/346] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:50:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:57.865166 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408a90a-46bd-4b24-b596-53ec90b641f5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:57.887207 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 178/347] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:50:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 15:50:57.888400 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b76ab6-41f7-4bc0-b824-4645ad3aecd0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:57.920461 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:58.032773 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b76ab6-41f7-4bc0-b824-4645ad3aecd0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:58.062561 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:58.064622 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408a90a-46bd-4b24-b596-53ec90b641f5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:58.082260 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c05874-9ca2-47f1-a397-c027c2b150f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:58.083477 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c05874-9ca2-47f1-a397-c027c2b150f6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:58.083894 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c05874-9ca2-47f1-a397-c027c2b150f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:58.084181 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c05874-9ca2-47f1-a397-c027c2b150f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:58.120480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5b319-cf7d-4c5d-9a4b-eda656c1dc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:58.121244 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5b319-cf7d-4c5d-9a4b-eda656c1dc04 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:58.121715 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5b319-cf7d-4c5d-9a4b-eda656c1dc04 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:58.122063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5b319-cf7d-4c5d-9a4b-eda656c1dc04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:58.183857 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c05874-9ca2-47f1-a397-c027c2b150f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:58.187881 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c05874-9ca2-47f1-a397-c027c2b150f6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:58.224525 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff5b319-cf7d-4c5d-9a4b-eda656c1dc04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:58.227223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff5b319-cf7d-4c5d-9a4b-eda656c1dc04 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:58.267019 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:58.392867 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 170/348] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:50:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 15:50:58.393847 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5b319-cf7d-4c5d-9a4b-eda656c1dc04 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:58.398365 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 179/349] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:50:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 15:50:58.400015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21a0a1-48c5-4914-b9a6-bcf92fb66d29 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:58.400156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21a0a1-48c5-4914-b9a6-bcf92fb66d29 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:58.400393 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21a0a1-48c5-4914-b9a6-bcf92fb66d29 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:58.401848 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 180/350] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:58.402993 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a384f9e-d057-4a44-8da1-90b7b7577fb3 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:58.403336 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a384f9e-d057-4a44-8da1-90b7b7577fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:58.403615 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a384f9e-d057-4a44-8da1-90b7b7577fb3 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:58.404858 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 181/351] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:58.405556 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a384f9e-d057-4a44-8da1-90b7b7577fb3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:58.441726 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:58.538517 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5b319-cf7d-4c5d-9a4b-eda656c1dc04 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:58.556754 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a384f9e-d057-4a44-8da1-90b7b7577fb3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:58.559926 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:58.587137 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b5512-32dc-4402-b254-5854eeaedff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:58.589242 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b5512-32dc-4402-b254-5854eeaedff4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:58.589994 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b5512-32dc-4402-b254-5854eeaedff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:58.590461 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b5512-32dc-4402-b254-5854eeaedff4 None admin] PATH_INFO: `/v3/projects/8edda125d8bb41498eca65801bd10882` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:58.592520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571b5512-32dc-4402-b254-5854eeaedff4 None admin] RBAC: Authorizing `identity:get_project(project_id=8edda125d8bb41498eca65801bd10882)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:58.594991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571b5512-32dc-4402-b254-5854eeaedff4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:58.597931 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 171/352] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:50:58 2022] GET /identity/v3/projects/8edda125d8bb41498eca65801bd10882 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:58.603136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96276892-ffd0-458a-97eb-963f7726a8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:58.604589 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c73304-5eaf-48d7-a53d-439b14d1beec None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:58.604589 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c73304-5eaf-48d7-a53d-439b14d1beec None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:58.604784 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96276892-ffd0-458a-97eb-963f7726a8c3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:58.604865 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c73304-5eaf-48d7-a53d-439b14d1beec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:58.604941 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96276892-ffd0-458a-97eb-963f7726a8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:58.604941 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96276892-ffd0-458a-97eb-963f7726a8c3 None admin] PATH_INFO: `/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:58.606100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96276892-ffd0-458a-97eb-963f7726a8c3 None admin] RBAC: Authorizing `identity:get_project(project_id=c9f5c8ab86d44f42af60e2d8442a4f3d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:58.607595 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96276892-ffd0-458a-97eb-963f7726a8c3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:58.614693 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 182/353] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:50:58 2022] GET /identity/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:58.615486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b93ff-1146-4053-8e17-25e7f9ad3404 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:58.615779 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b93ff-1146-4053-8e17-25e7f9ad3404 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:58.616059 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b93ff-1146-4053-8e17-25e7f9ad3404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:58.624335 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-70c73304-5eaf-48d7-a53d-439b14d1beec None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:58.625647 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-70c73304-5eaf-48d7-a53d-439b14d1beec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:58.634892 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d70b93ff-1146-4053-8e17-25e7f9ad3404 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:58.636109 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d70b93ff-1146-4053-8e17-25e7f9ad3404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:58.718489 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 172/354] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:58.719695 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c73304-5eaf-48d7-a53d-439b14d1beec None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:58.727369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 183/355] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:58.735169 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460c347-e268-4154-bc2b-bad3e6d3d44c None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:58.735781 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460c347-e268-4154-bc2b-bad3e6d3d44c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:58.735781 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460c347-e268-4154-bc2b-bad3e6d3d44c None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:58.737676 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 184/356] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:50:58 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:50:58.739292 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460c347-e268-4154-bc2b-bad3e6d3d44c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:58.767941 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:58.860634 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c73304-5eaf-48d7-a53d-439b14d1beec None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:58.876555 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:58.901284 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460c347-e268-4154-bc2b-bad3e6d3d44c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:58.905175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2966f-9aff-43d6-8dab-75a773f8e056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:58.906121 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2966f-9aff-43d6-8dab-75a773f8e056 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:58.906404 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2966f-9aff-43d6-8dab-75a773f8e056 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:58.906680 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2966f-9aff-43d6-8dab-75a773f8e056 None admin] PATH_INFO: `/v3/projects/8edda125d8bb41498eca65801bd10882` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:58.907937 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e2966f-9aff-43d6-8dab-75a773f8e056 None admin] RBAC: Authorizing `identity:get_project(project_id=8edda125d8bb41498eca65801bd10882)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:58.911169 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e2966f-9aff-43d6-8dab-75a773f8e056 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:58.913130 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 173/357] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:50:58 2022] GET /identity/v3/projects/8edda125d8bb41498eca65801bd10882 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:58.913801 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2966f-9aff-43d6-8dab-75a773f8e056 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:58.959867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e0e0f-349b-4e26-af7b-a9d6636c7023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:58.961096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e0e0f-349b-4e26-af7b-a9d6636c7023 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:58.961444 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e0e0f-349b-4e26-af7b-a9d6636c7023 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:58.964503 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e0e0f-349b-4e26-af7b-a9d6636c7023 None admin] PATH_INFO: `/v3/groups/b7883f2649cd47f79cb62f0396bd0df7` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:58.971905 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e0e0f-349b-4e26-af7b-a9d6636c7023 None admin] RBAC: Authorizing `identity:get_group(group_id=b7883f2649cd47f79cb62f0396bd0df7)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:58.973565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e0e0f-349b-4e26-af7b-a9d6636c7023 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:58.975685 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 185/358] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 21 15:50:58 2022] GET /identity/v3/groups/b7883f2649cd47f79cb62f0396bd0df7 => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:58.976702 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea2727d-ebf7-4d3a-9166-6952eb076b60 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:58.976984 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea2727d-ebf7-4d3a-9166-6952eb076b60 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:58.977260 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea2727d-ebf7-4d3a-9166-6952eb076b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:59.001410 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7ea2727d-ebf7-4d3a-9166-6952eb076b60 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:50:59.003201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7ea2727d-ebf7-4d3a-9166-6952eb076b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:50:59.072443 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2966f-9aff-43d6-8dab-75a773f8e056 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:59.089664 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 186/359] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:50:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:50:59.090546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea2727d-ebf7-4d3a-9166-6952eb076b60 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:59.111565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:59.127434 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7bf198-38cb-4bb5-9f4d-cf43301a6c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:59.128350 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7bf198-38cb-4bb5-9f4d-cf43301a6c11 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:59.128631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7bf198-38cb-4bb5-9f4d-cf43301a6c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:59.128907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7bf198-38cb-4bb5-9f4d-cf43301a6c11 None admin] PATH_INFO: `/v3/groups/b7883f2649cd47f79cb62f0396bd0df7` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:59.131183 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7bf198-38cb-4bb5-9f4d-cf43301a6c11 None admin] RBAC: Authorizing `identity:get_group(group_id=b7883f2649cd47f79cb62f0396bd0df7)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:59.132654 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7bf198-38cb-4bb5-9f4d-cf43301a6c11 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:59.137954 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 174/360] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 21 15:50:58 2022] GET /identity/v3/groups/b7883f2649cd47f79cb62f0396bd0df7 => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:59.137954 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7bf198-38cb-4bb5-9f4d-cf43301a6c11 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:59.197801 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:59.236664 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea2727d-ebf7-4d3a-9166-6952eb076b60 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:59.260747 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7bf198-38cb-4bb5-9f4d-cf43301a6c11 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:59.287361 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90bcc0d-d9b3-4606-881a-dc13f83351c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:59.288173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90bcc0d-d9b3-4606-881a-dc13f83351c8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:59.288458 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90bcc0d-d9b3-4606-881a-dc13f83351c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:59.288748 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90bcc0d-d9b3-4606-881a-dc13f83351c8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:59.300594 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90bcc0d-d9b3-4606-881a-dc13f83351c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:59.306185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c401c3-fceb-4ba9-9311-9c0176aec0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:59.307001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c401c3-fceb-4ba9-9311-9c0176aec0bd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:59.307279 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c401c3-fceb-4ba9-9311-9c0176aec0bd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:59.307552 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c401c3-fceb-4ba9-9311-9c0176aec0bd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:59.308416 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e90bcc0d-d9b3-4606-881a-dc13f83351c8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 15:50:59.308416 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:59.308416 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:59.308416 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:59.308416 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:59.308416 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:59.308416 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:59.308416 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:59.308416 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:59.308416 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:59.308416 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:59.308416 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:59.308416 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:59.308416 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:59.308416 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:59.308416 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:59.309433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:59.309433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:59.309433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:59.309433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:59.309433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:59.309433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:59.309433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:59.309433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:59.309433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:59.309433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:59.309433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:59.309433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:59.309433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:59.309433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:59.309433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:59.309433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:59.310625 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:59.310625 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:59.310625 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:59.310625 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:59.310625 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:59.310625 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:59.310625 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:59.310625 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:59.310625 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:59.310625 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:59.310625 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 15:50:59.310625 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:50:59.314001 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 187/361] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:50:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:59.315405 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90bcc0d-d9b3-4606-881a-dc13f83351c8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:59.321021 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c401c3-fceb-4ba9-9311-9c0176aec0bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:59.329425 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-87c401c3-fceb-4ba9-9311-9c0176aec0bd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 15:50:59.329425 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:59.329425 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:59.329425 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:59.329425 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:59.329425 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:59.329425 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:59.329425 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:59.329425 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:59.329425 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:59.329425 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:59.329425 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:59.329425 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:59.329425 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:59.329425 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:59.329425 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:59.330587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:59.330587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:59.330587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:59.330587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:59.330587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:59.330587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:59.330587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:59.330587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:59.330587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:59.330587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:59.330587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:59.330587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:59.330587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:59.330587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:59.330587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:59.330587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:59.332923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:59.332923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:59.332923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:59.332923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:59.332923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:59.332923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:59.332923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:59.332923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:59.332923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:59.332923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:59.332923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 15:50:59.332923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:50:59.332923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 175/362] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 15:50:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:59.332923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c401c3-fceb-4ba9-9311-9c0176aec0bd None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:59.426325 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:59.472770 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c401c3-fceb-4ba9-9311-9c0176aec0bd None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:59.474504 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90bcc0d-d9b3-4606-881a-dc13f83351c8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:59.487229 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:59.518576 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e3f7a-170f-4ad5-aaa9-4fecf116762a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:59.519488 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e3f7a-170f-4ad5-aaa9-4fecf116762a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:59.519888 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e3f7a-170f-4ad5-aaa9-4fecf116762a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:59.520292 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e3f7a-170f-4ad5-aaa9-4fecf116762a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:59.521756 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e3f7a-170f-4ad5-aaa9-4fecf116762a None admin] RBAC: Adding query filter params (group.id=b7883f2649cd47f79cb62f0396bd0df7, role.id=54195e660fc349978b39d5794765a095, scope.project.id=8edda125d8bb41498eca65801bd10882) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:59.522792 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e3f7a-170f-4ad5-aaa9-4fecf116762a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:59.524299 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e3f7a-170f-4ad5-aaa9-4fecf116762a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:59.530835 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0379b8-681c-49fc-bafc-373968c8b485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:59.531532 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 176/363] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jul 21 15:50:59 2022] GET /identity/v3/role_assignments?group.id=b7883f2649cd47f79cb62f0396bd0df7&scope.project.id=8edda125d8bb41498eca65801bd10882&role.id=54195e660fc349978b39d5794765a095 => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:59.531779 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0379b8-681c-49fc-bafc-373968c8b485 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:59.532072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0379b8-681c-49fc-bafc-373968c8b485 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:59.532214 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e3f7a-170f-4ad5-aaa9-4fecf116762a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:59.532522 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0379b8-681c-49fc-bafc-373968c8b485 None admin] PATH_INFO: `/v3/groups/b7883f2649cd47f79cb62f0396bd0df7` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:59.534500 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0379b8-681c-49fc-bafc-373968c8b485 None admin] RBAC: Authorizing `identity:get_group(group_id=b7883f2649cd47f79cb62f0396bd0df7)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:59.536610 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0379b8-681c-49fc-bafc-373968c8b485 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:59.539207 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 188/364] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 21 15:50:59 2022] GET /identity/v3/groups/b7883f2649cd47f79cb62f0396bd0df7 => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:59.540004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0379b8-681c-49fc-bafc-373968c8b485 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:59.694988 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0379b8-681c-49fc-bafc-373968c8b485 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:59.709118 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e3f7a-170f-4ad5-aaa9-4fecf116762a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:59.743412 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337336d9-f39f-4409-8218-747b4a286f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:59.744009 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337336d9-f39f-4409-8218-747b4a286f50 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:59.744290 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337336d9-f39f-4409-8218-747b4a286f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:59.744566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337336d9-f39f-4409-8218-747b4a286f50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:59.745422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337336d9-f39f-4409-8218-747b4a286f50 None admin] RBAC: Adding query filter params (group.id=b7883f2649cd47f79cb62f0396bd0df7, role.id=54195e660fc349978b39d5794765a095, scope.project.id=c9f5c8ab86d44f42af60e2d8442a4f3d) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:50:59.745804 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337336d9-f39f-4409-8218-747b4a286f50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:50:59.747277 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337336d9-f39f-4409-8218-747b4a286f50 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:50:59.754782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 189/365] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jul 21 15:50:59 2022] GET /identity/v3/role_assignments?group.id=b7883f2649cd47f79cb62f0396bd0df7&scope.project.id=c9f5c8ab86d44f42af60e2d8442a4f3d&role.id=54195e660fc349978b39d5794765a095 => generated 260 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:50:59.757625 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337336d9-f39f-4409-8218-747b4a286f50 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:59.762668 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:59.773533 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada9a88e-1eb2-430b-b5df-c11a1925002c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:50:59.774697 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9a88e-1eb2-430b-b5df-c11a1925002c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:50:59.775086 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9a88e-1eb2-430b-b5df-c11a1925002c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:50:59.775470 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9a88e-1eb2-430b-b5df-c11a1925002c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:50:59.790376 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada9a88e-1eb2-430b-b5df-c11a1925002c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:50:59.799695 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ada9a88e-1eb2-430b-b5df-c11a1925002c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:50:59.799695 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:50:59.799695 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:50:59.799695 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:50:59.799695 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:50:59.799695 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:50:59.799695 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:50:59.799695 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:50:59.799695 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:50:59.799695 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:50:59.799695 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:50:59.799695 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:50:59.799695 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:50:59.799695 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:50:59.799695 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:50:59.799695 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:50:59.800714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:50:59.800714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:50:59.800714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:50:59.800714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:50:59.800714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:50:59.800714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:50:59.800714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:50:59.800714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:50:59.800714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:50:59.800714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:50:59.800714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:50:59.800714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:50:59.800714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:50:59.800714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:50:59.800714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:50:59.800714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:50:59.801629 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:50:59.801629 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:50:59.801629 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:50:59.801629 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:50:59.801629 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:50:59.801629 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:50:59.801629 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:50:59.801629 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:50:59.801629 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:50:59.801629 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:50:59.801629 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:50:59.801629 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:50:59.801629 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 177/366] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:50:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:50:59.803063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada9a88e-1eb2-430b-b5df-c11a1925002c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:50:59.863830 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:50:59.954777 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337336d9-f39f-4409-8218-747b4a286f50 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:50:59.978630 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada9a88e-1eb2-430b-b5df-c11a1925002c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:00.011774 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59ecb1-3a0c-421b-9ae7-4ba7dd78622f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:00.012780 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59ecb1-3a0c-421b-9ae7-4ba7dd78622f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:00.013254 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59ecb1-3a0c-421b-9ae7-4ba7dd78622f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:00.013593 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59ecb1-3a0c-421b-9ae7-4ba7dd78622f None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:00.014378 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b59ecb1-3a0c-421b-9ae7-4ba7dd78622f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:00.014736 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b59ecb1-3a0c-421b-9ae7-4ba7dd78622f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:00.016137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b59ecb1-3a0c-421b-9ae7-4ba7dd78622f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:00.023237 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc34ed4-06c3-46af-81c9-de9ed8bad4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:00.024044 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc34ed4-06c3-46af-81c9-de9ed8bad4cc None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:00.024332 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc34ed4-06c3-46af-81c9-de9ed8bad4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:00.024610 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc34ed4-06c3-46af-81c9-de9ed8bad4cc None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:00.025388 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc34ed4-06c3-46af-81c9-de9ed8bad4cc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:00.025719 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc34ed4-06c3-46af-81c9-de9ed8bad4cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:00.027151 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc34ed4-06c3-46af-81c9-de9ed8bad4cc None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:00.028834 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 190/367] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 15:50:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:00.029754 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59ecb1-3a0c-421b-9ae7-4ba7dd78622f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:00.044519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 178/368] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:50:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:00.045483 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc34ed4-06c3-46af-81c9-de9ed8bad4cc None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:00.119666 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:00.160449 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59ecb1-3a0c-421b-9ae7-4ba7dd78622f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:00.202494 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:00.213467 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc34ed4-06c3-46af-81c9-de9ed8bad4cc None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:00.229994 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd8930d-bfc3-4239-944a-8224be3ebcef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:00.231531 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8930d-bfc3-4239-944a-8224be3ebcef None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:00.232736 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8930d-bfc3-4239-944a-8224be3ebcef None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:00.233455 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8930d-bfc3-4239-944a-8224be3ebcef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:00.234685 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd8930d-bfc3-4239-944a-8224be3ebcef None admin] RBAC: Adding query filter params (group.id=b7883f2649cd47f79cb62f0396bd0df7, role.id=85dcd5509fca4911881d45baf5f1d5a5, scope.project.id=8edda125d8bb41498eca65801bd10882) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:00.235188 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd8930d-bfc3-4239-944a-8224be3ebcef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:00.237965 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd8930d-bfc3-4239-944a-8224be3ebcef None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:00.251599 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 191/369] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jul 21 15:51:00 2022] GET /identity/v3/role_assignments?group.id=b7883f2649cd47f79cb62f0396bd0df7&scope.project.id=8edda125d8bb41498eca65801bd10882&role.id=85dcd5509fca4911881d45baf5f1d5a5 => generated 260 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:00.253702 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd8930d-bfc3-4239-944a-8224be3ebcef None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:00.270746 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9383d-b33c-4ee3-9570-14bcefc2ad2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:00.271829 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9383d-b33c-4ee3-9570-14bcefc2ad2a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:00.272388 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9383d-b33c-4ee3-9570-14bcefc2ad2a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:00.272870 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9383d-b33c-4ee3-9570-14bcefc2ad2a None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:00.273955 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f9383d-b33c-4ee3-9570-14bcefc2ad2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:00.274544 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f9383d-b33c-4ee3-9570-14bcefc2ad2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:00.276436 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f9383d-b33c-4ee3-9570-14bcefc2ad2a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:00.287461 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 179/370] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:51:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:00.288779 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9383d-b33c-4ee3-9570-14bcefc2ad2a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:00.405611 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd8930d-bfc3-4239-944a-8224be3ebcef None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:00.436810 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:00.444505 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7bc617-617a-4fea-9610-0c894ed6e885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:00.445451 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bc617-617a-4fea-9610-0c894ed6e885 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:00.445793 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bc617-617a-4fea-9610-0c894ed6e885 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:00.446072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bc617-617a-4fea-9610-0c894ed6e885 None admin] PATH_INFO: `/v3/users/71fd9dac8a29493bb9b393d3e2680e38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:00.447329 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7bc617-617a-4fea-9610-0c894ed6e885 None admin] RBAC: Authorizing `identity:get_user(user_id=71fd9dac8a29493bb9b393d3e2680e38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:00.448817 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7bc617-617a-4fea-9610-0c894ed6e885 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:00.450791 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 192/371] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:51:00 2022] GET /identity/v3/users/71fd9dac8a29493bb9b393d3e2680e38 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:00.451336 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7bc617-617a-4fea-9610-0c894ed6e885 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:00.539691 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9383d-b33c-4ee3-9570-14bcefc2ad2a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:00.569734 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7bc617-617a-4fea-9610-0c894ed6e885 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:00.609379 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0829e-52e8-440e-8c35-f6892d7b458a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:00.610233 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0829e-52e8-440e-8c35-f6892d7b458a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:00.610527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0829e-52e8-440e-8c35-f6892d7b458a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:00.610809 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0829e-52e8-440e-8c35-f6892d7b458a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:00.642970 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b94153-2da1-4efa-b55f-ddd1abbf620c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:00.643657 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b94153-2da1-4efa-b55f-ddd1abbf620c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:00.643952 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b94153-2da1-4efa-b55f-ddd1abbf620c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:00.644235 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b94153-2da1-4efa-b55f-ddd1abbf620c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:00.686790 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:00.691867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed0829e-52e8-440e-8c35-f6892d7b458a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:00.693393 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed0829e-52e8-440e-8c35-f6892d7b458a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:00.723861 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:00.809320 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 193/372] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:51:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 15:51:00.815404 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0829e-52e8-440e-8c35-f6892d7b458a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:00.874694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b94153-2da1-4efa-b55f-ddd1abbf620c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:00.876192 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b94153-2da1-4efa-b55f-ddd1abbf620c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:00.941409 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0829e-52e8-440e-8c35-f6892d7b458a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:00.943164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:00.997168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f17fe-530b-4597-a049-a039d155c567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:00.997881 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f17fe-530b-4597-a049-a039d155c567 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:00.998167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f17fe-530b-4597-a049-a039d155c567 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:00.998461 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f17fe-530b-4597-a049-a039d155c567 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:00.999304 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f17fe-530b-4597-a049-a039d155c567 None admin] RBAC: Adding query filter params (role.id=f4d0b371e4b140d9a9f6d9c6983c0ec8, user.id=71fd9dac8a29493bb9b393d3e2680e38, scope.system=all) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:00.999625 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f17fe-530b-4597-a049-a039d155c567 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:01.001109 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f17fe-530b-4597-a049-a039d155c567 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:01.020489 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 194/373] 10.222.0.52 () {56 vars in 1308 bytes} [Thu Jul 21 15:51:00 2022] GET /identity/v3/role_assignments?user.id=71fd9dac8a29493bb9b393d3e2680e38&scope.system=all&role.id=f4d0b371e4b140d9a9f6d9c6983c0ec8 => generated 510 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:01.021361 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f17fe-530b-4597-a049-a039d155c567 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:01.138504 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 180/374] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:51:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 850 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 15:51:01.144725 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b94153-2da1-4efa-b55f-ddd1abbf620c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:01.285728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f17fe-530b-4597-a049-a039d155c567 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:01.311146 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:01.376308 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d078b3-bebe-458c-8c23-47719fb7ad4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:01.377814 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d078b3-bebe-458c-8c23-47719fb7ad4a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:01.378100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d078b3-bebe-458c-8c23-47719fb7ad4a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:01.378378 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d078b3-bebe-458c-8c23-47719fb7ad4a None admin] PATH_INFO: `/v3/projects/f2cb644b2d994629b4b82065db94ebac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:01.380499 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d078b3-bebe-458c-8c23-47719fb7ad4a None admin] RBAC: Authorizing `identity:get_project(project_id=f2cb644b2d994629b4b82065db94ebac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:01.382005 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d078b3-bebe-458c-8c23-47719fb7ad4a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:01.383789 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 195/375] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:51:01 2022] GET /identity/v3/projects/f2cb644b2d994629b4b82065db94ebac => generated 344 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:01.389547 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d078b3-bebe-458c-8c23-47719fb7ad4a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:01.426064 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b94153-2da1-4efa-b55f-ddd1abbf620c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:01.461969 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:01.514117 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad065565-0aa2-4437-9b74-717db64ccdeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:01.514992 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad065565-0aa2-4437-9b74-717db64ccdeb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:01.515293 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad065565-0aa2-4437-9b74-717db64ccdeb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:01.515552 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad065565-0aa2-4437-9b74-717db64ccdeb None admin] PATH_INFO: `/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:01.518157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad065565-0aa2-4437-9b74-717db64ccdeb None admin] RBAC: Authorizing `identity:get_project(project_id=c9f5c8ab86d44f42af60e2d8442a4f3d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:01.519651 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad065565-0aa2-4437-9b74-717db64ccdeb None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:01.521753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 181/376] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:51:01 2022] GET /identity/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d => generated 302 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:51:01.528731 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad065565-0aa2-4437-9b74-717db64ccdeb None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:01.637681 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d078b3-bebe-458c-8c23-47719fb7ad4a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:01.704510 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13021153-66d4-48a9-bdc3-6e52fb98dda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:01.705349 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13021153-66d4-48a9-bdc3-6e52fb98dda2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:01.705654 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13021153-66d4-48a9-bdc3-6e52fb98dda2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:01.705940 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13021153-66d4-48a9-bdc3-6e52fb98dda2 None admin] PATH_INFO: `/v3/groups/8e23812fddc84ecfa4ea331a431b0e91` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:01.715099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13021153-66d4-48a9-bdc3-6e52fb98dda2 None admin] RBAC: Authorizing `identity:get_group(group_id=8e23812fddc84ecfa4ea331a431b0e91)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:01.716940 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13021153-66d4-48a9-bdc3-6e52fb98dda2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:01.718964 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 196/377] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 21 15:51:01 2022] GET /identity/v3/groups/8e23812fddc84ecfa4ea331a431b0e91 => generated 229 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:01.727193 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13021153-66d4-48a9-bdc3-6e52fb98dda2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:01.765694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad065565-0aa2-4437-9b74-717db64ccdeb None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:01.807072 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:01.814645 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b1f68-b404-4a3c-b35f-1e7d01dafcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:01.815910 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b1f68-b404-4a3c-b35f-1e7d01dafcbd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:01.816462 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b1f68-b404-4a3c-b35f-1e7d01dafcbd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:01.816755 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b1f68-b404-4a3c-b35f-1e7d01dafcbd None admin] PATH_INFO: `/v3/groups/b7883f2649cd47f79cb62f0396bd0df7` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:01.820070 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95b1f68-b404-4a3c-b35f-1e7d01dafcbd None admin] RBAC: Authorizing `identity:get_group(group_id=b7883f2649cd47f79cb62f0396bd0df7)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:01.821757 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95b1f68-b404-4a3c-b35f-1e7d01dafcbd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:01.823678 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 182/378] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 21 15:51:01 2022] GET /identity/v3/groups/b7883f2649cd47f79cb62f0396bd0df7 => generated 226 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:01.825130 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a94e7b-39b7-4168-b232-10bfc7711c82 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:01.825416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a94e7b-39b7-4168-b232-10bfc7711c82 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:01.825725 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a94e7b-39b7-4168-b232-10bfc7711c82 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:01.827044 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 183/379] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:01.837411 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a94e7b-39b7-4168-b232-10bfc7711c82 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:01.880798 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:01.896527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13021153-66d4-48a9-bdc3-6e52fb98dda2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:01.942634 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc3c9fd-9bf7-4285-82dc-17b6e56e1e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:01.943276 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3c9fd-9bf7-4285-82dc-17b6e56e1e7b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:01.943549 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3c9fd-9bf7-4285-82dc-17b6e56e1e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:01.943825 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3c9fd-9bf7-4285-82dc-17b6e56e1e7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:01.945496 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc3c9fd-9bf7-4285-82dc-17b6e56e1e7b None admin] RBAC: Adding query filter params (group.id=8e23812fddc84ecfa4ea331a431b0e91, role.id=475521b8a0174037b162f92022aea1a1, scope.project.id=f2cb644b2d994629b4b82065db94ebac) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:01.945846 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc3c9fd-9bf7-4285-82dc-17b6e56e1e7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:01.947405 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc3c9fd-9bf7-4285-82dc-17b6e56e1e7b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:01.955869 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 197/380] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jul 21 15:51:01 2022] GET /identity/v3/role_assignments?group.id=8e23812fddc84ecfa4ea331a431b0e91&scope.project.id=f2cb644b2d994629b4b82065db94ebac&role.id=475521b8a0174037b162f92022aea1a1 => generated 260 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:01.957404 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcab4bc-aef0-48c7-9f39-943d75231c21 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:01.958073 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcab4bc-aef0-48c7-9f39-943d75231c21 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:01.958358 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcab4bc-aef0-48c7-9f39-943d75231c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:01.985886 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8dcab4bc-aef0-48c7-9f39-943d75231c21 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:01.985886 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8dcab4bc-aef0-48c7-9f39-943d75231c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:01.991626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a94e7b-39b7-4168-b232-10bfc7711c82 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:02.037393 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931684ec-dcd0-401a-becb-2c7364fc020f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:02.038100 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931684ec-dcd0-401a-becb-2c7364fc020f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:02.038849 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931684ec-dcd0-401a-becb-2c7364fc020f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:02.039641 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931684ec-dcd0-401a-becb-2c7364fc020f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:02.040545 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931684ec-dcd0-401a-becb-2c7364fc020f None admin] RBAC: Adding query filter params (group.id=b7883f2649cd47f79cb62f0396bd0df7, role.id=85dcd5509fca4911881d45baf5f1d5a5, scope.project.id=c9f5c8ab86d44f42af60e2d8442a4f3d) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:02.040910 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931684ec-dcd0-401a-becb-2c7364fc020f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:02.042413 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931684ec-dcd0-401a-becb-2c7364fc020f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:02.050123 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 184/381] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jul 21 15:51:01 2022] GET /identity/v3/role_assignments?group.id=b7883f2649cd47f79cb62f0396bd0df7&scope.project.id=c9f5c8ab86d44f42af60e2d8442a4f3d&role.id=85dcd5509fca4911881d45baf5f1d5a5 => generated 260 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:02.139830 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 198/382] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:51:02.171268 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931684ec-dcd0-401a-becb-2c7364fc020f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:02.271921 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:02.339327 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931684ec-dcd0-401a-becb-2c7364fc020f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:02.394866 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392257b7-c783-47fb-a889-c9e357aa12bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:02.396185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392257b7-c783-47fb-a889-c9e357aa12bc None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:02.396561 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392257b7-c783-47fb-a889-c9e357aa12bc None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:02.396895 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392257b7-c783-47fb-a889-c9e357aa12bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:02.417602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392257b7-c783-47fb-a889-c9e357aa12bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:02.426270 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-392257b7-c783-47fb-a889-c9e357aa12bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:51:02.426270 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:02.426270 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:02.426270 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:02.426270 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:02.426270 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:02.426270 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:02.426270 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:02.426270 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:02.426270 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:02.426270 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:02.426270 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:02.426270 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:02.426270 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:02.426270 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:02.426270 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:02.427240 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:02.427240 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:02.427240 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:02.427240 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:02.427240 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:02.427240 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:02.427240 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:02.427240 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:02.427240 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:02.427240 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:02.427240 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:02.427240 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:02.427240 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:02.427240 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:02.427240 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:02.427240 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:02.428492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:02.428492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:02.428492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:02.428492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:02.428492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:02.428492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:02.428492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:02.428492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:02.428492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:02.428492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:02.428492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:51:02.428492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:02.428492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 185/383] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:51:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:02.433401 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcab4bc-aef0-48c7-9f39-943d75231c21 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:02.530323 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:02.550446 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcab4bc-aef0-48c7-9f39-943d75231c21 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:02.566085 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710b2b8-5214-4377-b096-83081ffcf705 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:02.566518 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710b2b8-5214-4377-b096-83081ffcf705 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:02.567066 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710b2b8-5214-4377-b096-83081ffcf705 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:02.569288 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 186/384] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 15:51:02.574576 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e4e6f-03b1-4fb4-b3ba-45de78577d7f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:02.574857 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e4e6f-03b1-4fb4-b3ba-45de78577d7f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:02.575137 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e4e6f-03b1-4fb4-b3ba-45de78577d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:02.593928 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b647191-4581-44c4-a2ab-89aa70e3c80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:02.594619 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b647191-4581-44c4-a2ab-89aa70e3c80b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:02.594902 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b647191-4581-44c4-a2ab-89aa70e3c80b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:02.595178 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b647191-4581-44c4-a2ab-89aa70e3c80b None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:02.596016 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b647191-4581-44c4-a2ab-89aa70e3c80b None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:02.596324 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b647191-4581-44c4-a2ab-89aa70e3c80b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:02.597814 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b647191-4581-44c4-a2ab-89aa70e3c80b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:02.602105 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ad2e4e6f-03b1-4fb4-b3ba-45de78577d7f None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:02.603323 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ad2e4e6f-03b1-4fb4-b3ba-45de78577d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:02.608185 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 199/385] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 15:51:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:02.614694 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b647191-4581-44c4-a2ab-89aa70e3c80b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:02.745956 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b647191-4581-44c4-a2ab-89aa70e3c80b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:02.749602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 187/386] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 15:51:02.807544 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e4e6f-03b1-4fb4-b3ba-45de78577d7f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:02.810992 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab3270-9f60-49f6-8e3e-2eaa65e45329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:02.817468 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab3270-9f60-49f6-8e3e-2eaa65e45329 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:02.817835 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab3270-9f60-49f6-8e3e-2eaa65e45329 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:02.818120 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab3270-9f60-49f6-8e3e-2eaa65e45329 None admin] PATH_INFO: `/v3/groups/b7883f2649cd47f79cb62f0396bd0df7` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:02.821532 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab3270-9f60-49f6-8e3e-2eaa65e45329 None admin] RBAC: Authorizing `identity:get_group(group_id=b7883f2649cd47f79cb62f0396bd0df7)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:02.823070 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab3270-9f60-49f6-8e3e-2eaa65e45329 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:02.824774 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 200/387] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 21 15:51:02 2022] GET /identity/v3/groups/b7883f2649cd47f79cb62f0396bd0df7 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:51:02.830815 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab3270-9f60-49f6-8e3e-2eaa65e45329 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:02.861942 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:02.914506 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:02.989600 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab3270-9f60-49f6-8e3e-2eaa65e45329 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:03.030131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f03b2a-9c82-435a-9c5d-fced584dbf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:03.030131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f03b2a-9c82-435a-9c5d-fced584dbf8e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:03.030131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f03b2a-9c82-435a-9c5d-fced584dbf8e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:03.030131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f03b2a-9c82-435a-9c5d-fced584dbf8e None admin] PATH_INFO: `/v3/projects/8edda125d8bb41498eca65801bd10882` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:03.030551 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f03b2a-9c82-435a-9c5d-fced584dbf8e None admin] RBAC: Authorizing `identity:get_project(project_id=8edda125d8bb41498eca65801bd10882)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:03.030918 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f03b2a-9c82-435a-9c5d-fced584dbf8e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:03.033472 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 201/388] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:51:02 2022] GET /identity/v3/projects/8edda125d8bb41498eca65801bd10882 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:03.038905 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f03b2a-9c82-435a-9c5d-fced584dbf8e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:03.056148 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e4e6f-03b1-4fb4-b3ba-45de78577d7f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:03.113002 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1b73e-de33-4ba9-acf3-50d9df687e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:03.113765 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1b73e-de33-4ba9-acf3-50d9df687e00 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:03.114054 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1b73e-de33-4ba9-acf3-50d9df687e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:03.114335 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1b73e-de33-4ba9-acf3-50d9df687e00 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:03.136678 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1b73e-de33-4ba9-acf3-50d9df687e00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:03.155468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-fdd1b73e-de33-4ba9-acf3-50d9df687e00 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:51:03.155468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:03.155468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:03.155468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:03.155468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:03.155468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:03.155468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:03.155468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:03.155468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:03.155468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:03.155468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:03.155468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:03.155468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:03.155468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:03.155468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:03.155468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:03.156470 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:03.156470 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:03.156470 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:03.156470 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:03.156470 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:03.156470 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:03.156470 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:03.156470 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:03.156470 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:03.156470 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:03.156470 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:03.156470 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:03.156470 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:03.156470 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:03.156470 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:03.156470 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:03.157309 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:03.157309 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:03.157309 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:03.157309 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:03.157309 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:03.157309 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:03.157309 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:03.157309 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:03.157309 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:03.157309 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:03.157309 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:51:03.157309 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:03.159186 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 188/389] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:51:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:51:03.166625 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1b73e-de33-4ba9-acf3-50d9df687e00 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:03.177934 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:03.216136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f03b2a-9c82-435a-9c5d-fced584dbf8e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:03.276292 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b142bf7d-a36c-43c8-ba04-3376472da1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:03.277652 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b142bf7d-a36c-43c8-ba04-3376472da1e1 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:03.277939 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b142bf7d-a36c-43c8-ba04-3376472da1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:03.278487 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b142bf7d-a36c-43c8-ba04-3376472da1e1 None admin] PATH_INFO: `/v3/projects/8edda125d8bb41498eca65801bd10882/groups/b7883f2649cd47f79cb62f0396bd0df7/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:03.281146 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b142bf7d-a36c-43c8-ba04-3376472da1e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=8edda125d8bb41498eca65801bd10882, group_id=b7883f2649cd47f79cb62f0396bd0df7, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:03.283252 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b142bf7d-a36c-43c8-ba04-3376472da1e1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:03.302247 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 202/390] 10.222.0.52 () {58 vars in 1442 bytes} [Thu Jul 21 15:51:03 2022] PUT /identity/v3/projects/8edda125d8bb41498eca65801bd10882/groups/b7883f2649cd47f79cb62f0396bd0df7/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 15:51:03.303772 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cbc77-db6a-4991-8c16-f1a1c398a8a6 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:03.304180 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cbc77-db6a-4991-8c16-f1a1c398a8a6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:03.304527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cbc77-db6a-4991-8c16-f1a1c398a8a6 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:03.309829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 203/391] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:03.319638 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427c7c3-d09e-46fd-b1fd-384963621f13 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:03.319638 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427c7c3-d09e-46fd-b1fd-384963621f13 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:03.319638 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427c7c3-d09e-46fd-b1fd-384963621f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:03.366136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3427c7c3-d09e-46fd-b1fd-384963621f13 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:03.368499 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3427c7c3-d09e-46fd-b1fd-384963621f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:03.377422 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1b73e-de33-4ba9-acf3-50d9df687e00 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:03.404186 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:03.458959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58037716-5345-4ea3-bd48-4ab4c522d76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:03.459657 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58037716-5345-4ea3-bd48-4ab4c522d76f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:03.459947 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58037716-5345-4ea3-bd48-4ab4c522d76f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:03.460236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58037716-5345-4ea3-bd48-4ab4c522d76f None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:03.461030 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58037716-5345-4ea3-bd48-4ab4c522d76f None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:03.461345 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58037716-5345-4ea3-bd48-4ab4c522d76f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:03.462912 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58037716-5345-4ea3-bd48-4ab4c522d76f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:03.467374 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 204/392] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:51:03.480528 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 189/393] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 15:51:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:03.485806 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427c7c3-d09e-46fd-b1fd-384963621f13 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:03.507217 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58037716-5345-4ea3-bd48-4ab4c522d76f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:03.580819 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:03.619666 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427c7c3-d09e-46fd-b1fd-384963621f13 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:03.662064 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3296f12d-095b-41fc-bada-bbd01cd57bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:03.662943 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296f12d-095b-41fc-bada-bbd01cd57bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:03.663223 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296f12d-095b-41fc-bada-bbd01cd57bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:03.663491 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296f12d-095b-41fc-bada-bbd01cd57bb8 None admin] PATH_INFO: `/v3/groups/b7883f2649cd47f79cb62f0396bd0df7` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:03.665274 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3296f12d-095b-41fc-bada-bbd01cd57bb8 None admin] RBAC: Authorizing `identity:get_group(group_id=b7883f2649cd47f79cb62f0396bd0df7)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:03.666792 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3296f12d-095b-41fc-bada-bbd01cd57bb8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:03.669415 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 205/394] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 21 15:51:03 2022] GET /identity/v3/groups/b7883f2649cd47f79cb62f0396bd0df7 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:03.675450 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3296f12d-095b-41fc-bada-bbd01cd57bb8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:03.710550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58037716-5345-4ea3-bd48-4ab4c522d76f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:03.788263 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3296f12d-095b-41fc-bada-bbd01cd57bb8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:03.827991 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95106ebd-5a5d-40be-b80b-f9b4cd912c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:03.828867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95106ebd-5a5d-40be-b80b-f9b4cd912c85 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:03.829151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95106ebd-5a5d-40be-b80b-f9b4cd912c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:03.829526 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95106ebd-5a5d-40be-b80b-f9b4cd912c85 None admin] PATH_INFO: `/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:03.831059 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95106ebd-5a5d-40be-b80b-f9b4cd912c85 None admin] RBAC: Authorizing `identity:get_project(project_id=c9f5c8ab86d44f42af60e2d8442a4f3d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:03.832518 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95106ebd-5a5d-40be-b80b-f9b4cd912c85 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:03.834595 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 206/395] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:51:03 2022] GET /identity/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:51:03.838362 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd01ec-8702-47c5-b67d-b6d382fe928e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:03.839274 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd01ec-8702-47c5-b67d-b6d382fe928e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:03.839570 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd01ec-8702-47c5-b67d-b6d382fe928e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:03.839857 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd01ec-8702-47c5-b67d-b6d382fe928e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:03.840092 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95106ebd-5a5d-40be-b80b-f9b4cd912c85 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:03.855500 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:03.872227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd01ec-8702-47c5-b67d-b6d382fe928e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:03.888225 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-3acd01ec-8702-47c5-b67d-b6d382fe928e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 15:51:03.888225 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:03.888225 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:03.888225 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:03.888225 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:03.888225 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:03.888225 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:03.888225 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:03.888225 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:03.888225 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:03.888225 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:03.888225 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:03.888225 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:03.888225 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:03.888225 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:03.888225 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:03.890446 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:03.890446 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:03.890446 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:03.890446 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:03.890446 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:03.890446 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:03.890446 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:03.890446 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:03.890446 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:03.890446 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:03.890446 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:03.890446 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:03.890446 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:03.890446 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:03.890446 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:03.890446 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:03.892876 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:03.892876 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:03.892876 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:03.892876 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:03.892876 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:03.892876 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:03.892876 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:03.892876 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:03.892876 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:03.892876 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:03.892876 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 15:51:03.892876 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:03.898000 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 190/396] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 15:51:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:03.905041 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd01ec-8702-47c5-b67d-b6d382fe928e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:03.957672 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95106ebd-5a5d-40be-b80b-f9b4cd912c85 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:03.995410 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:03.997509 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15847aef-acec-4034-aea6-39fa438544f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:03.999072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15847aef-acec-4034-aea6-39fa438544f9 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:03.999532 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15847aef-acec-4034-aea6-39fa438544f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:03.999969 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15847aef-acec-4034-aea6-39fa438544f9 None admin] PATH_INFO: `/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d/groups/b7883f2649cd47f79cb62f0396bd0df7/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:04.002350 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15847aef-acec-4034-aea6-39fa438544f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=c9f5c8ab86d44f42af60e2d8442a4f3d, group_id=b7883f2649cd47f79cb62f0396bd0df7, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:04.004581 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15847aef-acec-4034-aea6-39fa438544f9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:04.024479 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 207/397] 10.222.0.52 () {58 vars in 1442 bytes} [Thu Jul 21 15:51:03 2022] PUT /identity/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d/groups/b7883f2649cd47f79cb62f0396bd0df7/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 15:51:04.133890 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd01ec-8702-47c5-b67d-b6d382fe928e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:04.183891 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13008890-c691-42f8-8e09-95b183d4e612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:04.184982 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13008890-c691-42f8-8e09-95b183d4e612 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:04.185490 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13008890-c691-42f8-8e09-95b183d4e612 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:04.186021 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13008890-c691-42f8-8e09-95b183d4e612 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:04.187012 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13008890-c691-42f8-8e09-95b183d4e612 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:04.187527 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13008890-c691-42f8-8e09-95b183d4e612 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:04.189225 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13008890-c691-42f8-8e09-95b183d4e612 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:04.200031 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 191/398] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:51:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:04.207194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15847aef-acec-4034-aea6-39fa438544f9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:04.321991 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:04.381060 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15847aef-acec-4034-aea6-39fa438544f9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:04.429474 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a718c-8acc-4ede-a180-b14a59a453c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:04.430298 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a718c-8acc-4ede-a180-b14a59a453c6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:04.430578 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a718c-8acc-4ede-a180-b14a59a453c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:04.430849 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a718c-8acc-4ede-a180-b14a59a453c6 None admin] PATH_INFO: `/v3/groups/b7883f2649cd47f79cb62f0396bd0df7` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:04.432142 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12a718c-8acc-4ede-a180-b14a59a453c6 None admin] RBAC: Authorizing `identity:get_group(group_id=b7883f2649cd47f79cb62f0396bd0df7)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:04.433689 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12a718c-8acc-4ede-a180-b14a59a453c6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:04.435640 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 208/399] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 21 15:51:04 2022] GET /identity/v3/groups/b7883f2649cd47f79cb62f0396bd0df7 => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:51:04.441665 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13008890-c691-42f8-8e09-95b183d4e612 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:04.579067 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:04.596331 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13008890-c691-42f8-8e09-95b183d4e612 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:04.648721 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc15133-0aa8-405b-8a78-d00d41a9a150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:04.649627 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc15133-0aa8-405b-8a78-d00d41a9a150 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:04.649917 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc15133-0aa8-405b-8a78-d00d41a9a150 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:04.650193 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc15133-0aa8-405b-8a78-d00d41a9a150 None admin] PATH_INFO: `/v3/projects/8edda125d8bb41498eca65801bd10882` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:04.651482 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc15133-0aa8-405b-8a78-d00d41a9a150 None admin] RBAC: Authorizing `identity:get_project(project_id=8edda125d8bb41498eca65801bd10882)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:04.652959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc15133-0aa8-405b-8a78-d00d41a9a150 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:04.655385 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 192/400] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:51:04 2022] GET /identity/v3/projects/8edda125d8bb41498eca65801bd10882 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:04.661893 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a718c-8acc-4ede-a180-b14a59a453c6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:04.702513 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd7f9a-3b78-4f90-8139-db4c13a556d7 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:04.702806 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd7f9a-3b78-4f90-8139-db4c13a556d7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:04.703186 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd7f9a-3b78-4f90-8139-db4c13a556d7 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:04.706807 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 193/401] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 15:51:04.715740 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13afd89-c10a-4652-ad58-9a101f2666b0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:04.716042 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13afd89-c10a-4652-ad58-9a101f2666b0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:04.716715 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13afd89-c10a-4652-ad58-9a101f2666b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:04.750779 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b13afd89-c10a-4652-ad58-9a101f2666b0 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:04.752204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b13afd89-c10a-4652-ad58-9a101f2666b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:04.848417 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 194/402] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:51:04.850049 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a528643-e44b-4199-ac5a-6d0cd3d88289 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:04.850401 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a528643-e44b-4199-ac5a-6d0cd3d88289 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:04.850680 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a528643-e44b-4199-ac5a-6d0cd3d88289 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:04.852238 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 195/403] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:04.859009 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a718c-8acc-4ede-a180-b14a59a453c6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:04.867297 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6128890-feb9-404e-9516-c931fb062c95 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:04.867594 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6128890-feb9-404e-9516-c931fb062c95 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:04.867843 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6128890-feb9-404e-9516-c931fb062c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:04.888650 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c6128890-feb9-404e-9516-c931fb062c95 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:04.890409 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c6128890-feb9-404e-9516-c931fb062c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:04.913424 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd87721-070c-488c-a6ac-2c2e6472f47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:04.914504 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd87721-070c-488c-a6ac-2c2e6472f47b None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:04.914786 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd87721-070c-488c-a6ac-2c2e6472f47b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:04.915059 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd87721-070c-488c-a6ac-2c2e6472f47b None admin] PATH_INFO: `/v3/projects/8edda125d8bb41498eca65801bd10882/groups/b7883f2649cd47f79cb62f0396bd0df7/roles/85dcd5509fca4911881d45baf5f1d5a5` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:04.917557 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd87721-070c-488c-a6ac-2c2e6472f47b None admin] RBAC: Authorizing `identity:create_grant(project_id=8edda125d8bb41498eca65801bd10882, group_id=b7883f2649cd47f79cb62f0396bd0df7, role_id=85dcd5509fca4911881d45baf5f1d5a5)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:04.919076 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd87721-070c-488c-a6ac-2c2e6472f47b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:04.921688 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:04.943385 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 209/404] 10.222.0.52 () {58 vars in 1442 bytes} [Thu Jul 21 15:51:04 2022] PUT /identity/v3/projects/8edda125d8bb41498eca65801bd10882/groups/b7883f2649cd47f79cb62f0396bd0df7/roles/85dcd5509fca4911881d45baf5f1d5a5 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 15:51:04.949849 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd87721-070c-488c-a6ac-2c2e6472f47b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:05.017721 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 196/405] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 15:51:05.119219 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6128890-feb9-404e-9516-c931fb062c95 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:05.149232 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:05.154035 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd87721-070c-488c-a6ac-2c2e6472f47b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:05.203778 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9420fa5-e1eb-4fb9-804a-56cfff092234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:05.205660 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9420fa5-e1eb-4fb9-804a-56cfff092234 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:05.205951 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9420fa5-e1eb-4fb9-804a-56cfff092234 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:05.206225 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9420fa5-e1eb-4fb9-804a-56cfff092234 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:05.223675 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9420fa5-e1eb-4fb9-804a-56cfff092234 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:05.233618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d9420fa5-e1eb-4fb9-804a-56cfff092234 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:51:05.233618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:05.233618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:05.233618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:05.233618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:05.233618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:05.233618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:05.233618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:05.233618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:05.233618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:05.233618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:05.233618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:05.233618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:05.233618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:05.233618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:05.233618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:05.234594 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:05.234594 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:05.234594 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:05.234594 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:05.234594 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:05.234594 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:05.234594 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:05.234594 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:05.234594 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:05.234594 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:05.234594 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:05.234594 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:05.234594 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:05.234594 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:05.234594 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:05.234594 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:05.235406 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:05.235406 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:05.235406 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:05.235406 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:05.235406 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:05.235406 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:05.235406 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:05.235406 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:05.235406 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:05.235406 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:05.235406 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:51:05.235406 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:05.244858 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 210/406] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:51:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:05.249190 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9420fa5-e1eb-4fb9-804a-56cfff092234 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:05.318095 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:05.322165 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6128890-feb9-404e-9516-c931fb062c95 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:05.378308 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9420fa5-e1eb-4fb9-804a-56cfff092234 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:05.422823 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880eb6cc-228e-4887-a927-1a773821f432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:05.424733 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880eb6cc-228e-4887-a927-1a773821f432 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:05.426267 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fad2e0-dca0-4835-a213-5ce3e6309dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:05.426753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880eb6cc-228e-4887-a927-1a773821f432 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:05.427330 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880eb6cc-228e-4887-a927-1a773821f432 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:05.428350 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fad2e0-dca0-4835-a213-5ce3e6309dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:05.429345 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880eb6cc-228e-4887-a927-1a773821f432 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:05.429879 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880eb6cc-228e-4887-a927-1a773821f432 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:05.430360 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fad2e0-dca0-4835-a213-5ce3e6309dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:05.430846 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fad2e0-dca0-4835-a213-5ce3e6309dc8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:05.432526 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880eb6cc-228e-4887-a927-1a773821f432 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:05.454109 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 211/407] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:51:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:05.458165 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fad2e0-dca0-4835-a213-5ce3e6309dc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:05.459906 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880eb6cc-228e-4887-a927-1a773821f432 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:05.478644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-c5fad2e0-dca0-4835-a213-5ce3e6309dc8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 15:51:05.478644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:05.478644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:05.478644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:05.478644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:05.478644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:05.478644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:05.478644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:05.478644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:05.478644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:05.478644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:05.478644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:05.478644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:05.478644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:05.478644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:05.478644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:05.479618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:05.479618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:05.479618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:05.479618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:05.479618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:05.479618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:05.479618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:05.479618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:05.479618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:05.479618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:05.479618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:05.479618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:05.479618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:05.479618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:05.479618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:05.479618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:05.480482 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:05.480482 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:05.480482 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:05.480482 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:05.480482 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:05.480482 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:05.480482 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:05.480482 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:05.480482 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:05.480482 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:05.480482 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 15:51:05.480482 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:05.486836 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 197/408] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 15:51:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:51:05.492586 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fad2e0-dca0-4835-a213-5ce3e6309dc8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:05.609428 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880eb6cc-228e-4887-a927-1a773821f432 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:05.630806 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:05.663127 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b7b15-6fd8-43cf-81a0-0c2aa5efa3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:05.664019 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b7b15-6fd8-43cf-81a0-0c2aa5efa3a6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:05.664303 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b7b15-6fd8-43cf-81a0-0c2aa5efa3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:05.664585 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b7b15-6fd8-43cf-81a0-0c2aa5efa3a6 None admin] PATH_INFO: `/v3/groups/8e23812fddc84ecfa4ea331a431b0e91` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:05.666201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46b7b15-6fd8-43cf-81a0-0c2aa5efa3a6 None admin] RBAC: Authorizing `identity:get_group(group_id=8e23812fddc84ecfa4ea331a431b0e91)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:05.667677 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46b7b15-6fd8-43cf-81a0-0c2aa5efa3a6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:05.673936 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 212/409] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 21 15:51:05 2022] GET /identity/v3/groups/8e23812fddc84ecfa4ea331a431b0e91 => generated 229 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:05.677107 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b7b15-6fd8-43cf-81a0-0c2aa5efa3a6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:05.761942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:05.795989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b7b15-6fd8-43cf-81a0-0c2aa5efa3a6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:05.831422 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fad2e0-dca0-4835-a213-5ce3e6309dc8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:05.836317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d45d52-8400-492c-8558-f63910a05276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:05.837528 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d45d52-8400-492c-8558-f63910a05276 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:05.838038 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d45d52-8400-492c-8558-f63910a05276 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:05.839529 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d45d52-8400-492c-8558-f63910a05276 None admin] PATH_INFO: `/v3/projects/f2cb644b2d994629b4b82065db94ebac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:05.841165 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d45d52-8400-492c-8558-f63910a05276 None admin] RBAC: Authorizing `identity:get_project(project_id=f2cb644b2d994629b4b82065db94ebac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:05.842965 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d45d52-8400-492c-8558-f63910a05276 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:05.846452 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 213/410] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:51:05 2022] GET /identity/v3/projects/f2cb644b2d994629b4b82065db94ebac => generated 344 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:05.852998 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d45d52-8400-492c-8558-f63910a05276 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:05.919700 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a071f-5593-477b-b90d-5d1498a98d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:05.920308 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a071f-5593-477b-b90d-5d1498a98d38 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:05.920600 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a071f-5593-477b-b90d-5d1498a98d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:05.920885 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a071f-5593-477b-b90d-5d1498a98d38 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:05.921700 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26a071f-5593-477b-b90d-5d1498a98d38 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:05.922018 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26a071f-5593-477b-b90d-5d1498a98d38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:05.923501 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26a071f-5593-477b-b90d-5d1498a98d38 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:05.924033 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:05.945113 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 198/411] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:51:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 453 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:05.950446 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a071f-5593-477b-b90d-5d1498a98d38 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:05.985506 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d45d52-8400-492c-8558-f63910a05276 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:06.034563 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04dee6-4eab-4383-abec-94b55b7084b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:06.035512 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04dee6-4eab-4383-abec-94b55b7084b9 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:06.035802 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04dee6-4eab-4383-abec-94b55b7084b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:06.036082 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04dee6-4eab-4383-abec-94b55b7084b9 None admin] PATH_INFO: `/v3/projects/f2cb644b2d994629b4b82065db94ebac/groups/8e23812fddc84ecfa4ea331a431b0e91/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:06.038268 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d04dee6-4eab-4383-abec-94b55b7084b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2cb644b2d994629b4b82065db94ebac, group_id=8e23812fddc84ecfa4ea331a431b0e91, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:06.039836 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d04dee6-4eab-4383-abec-94b55b7084b9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:06.058478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 214/412] 10.222.0.52 () {58 vars in 1442 bytes} [Thu Jul 21 15:51:05 2022] PUT /identity/v3/projects/f2cb644b2d994629b4b82065db94ebac/groups/8e23812fddc84ecfa4ea331a431b0e91/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 15:51:06.060234 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181bd7ec-bb98-406f-87e4-451346f105cf None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:06.060678 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181bd7ec-bb98-406f-87e4-451346f105cf None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:06.061148 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181bd7ec-bb98-406f-87e4-451346f105cf None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:06.069963 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 215/413] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:06.072348 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575ee32-0baa-4cf5-b3ce-f0113e806e87 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:06.072701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575ee32-0baa-4cf5-b3ce-f0113e806e87 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:06.072915 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575ee32-0baa-4cf5-b3ce-f0113e806e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:06.090401 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a071f-5593-477b-b90d-5d1498a98d38 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:06.095522 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1575ee32-0baa-4cf5-b3ce-f0113e806e87 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:06.096243 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1575ee32-0baa-4cf5-b3ce-f0113e806e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:06.166079 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dedc0b0-1b7b-4a05-b418-dc1aa2de6141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:06.167638 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dedc0b0-1b7b-4a05-b418-dc1aa2de6141 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:06.168117 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dedc0b0-1b7b-4a05-b418-dc1aa2de6141 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:06.168601 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dedc0b0-1b7b-4a05-b418-dc1aa2de6141 None admin] PATH_INFO: `/v3/groups/b7883f2649cd47f79cb62f0396bd0df7` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:06.170232 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dedc0b0-1b7b-4a05-b418-dc1aa2de6141 None admin] RBAC: Authorizing `identity:get_group(group_id=b7883f2649cd47f79cb62f0396bd0df7)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:06.179243 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dedc0b0-1b7b-4a05-b418-dc1aa2de6141 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:06.181805 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 199/414] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 21 15:51:05 2022] GET /identity/v3/groups/b7883f2649cd47f79cb62f0396bd0df7 => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:51:06.187309 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dedc0b0-1b7b-4a05-b418-dc1aa2de6141 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:06.205606 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 216/415] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 15:51:06.211508 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:06.228700 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575ee32-0baa-4cf5-b3ce-f0113e806e87 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:06.280509 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:06.359361 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dedc0b0-1b7b-4a05-b418-dc1aa2de6141 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:06.359936 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575ee32-0baa-4cf5-b3ce-f0113e806e87 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:06.400612 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1474294-20f7-4193-9261-34612c71ad71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:06.402086 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1474294-20f7-4193-9261-34612c71ad71 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:06.402267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da3e37-acc4-4717-a24b-f8979f425820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:06.402768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1474294-20f7-4193-9261-34612c71ad71 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:06.403090 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da3e37-acc4-4717-a24b-f8979f425820 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:06.403382 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da3e37-acc4-4717-a24b-f8979f425820 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:06.403643 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1474294-20f7-4193-9261-34612c71ad71 None admin] PATH_INFO: `/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:06.403739 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da3e37-acc4-4717-a24b-f8979f425820 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:06.405374 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1474294-20f7-4193-9261-34612c71ad71 None admin] RBAC: Authorizing `identity:get_project(project_id=c9f5c8ab86d44f42af60e2d8442a4f3d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:06.407258 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1474294-20f7-4193-9261-34612c71ad71 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:06.409684 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 200/416] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:51:06 2022] GET /identity/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:51:06.421673 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1474294-20f7-4193-9261-34612c71ad71 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:06.421835 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da3e37-acc4-4717-a24b-f8979f425820 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:06.425098 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-30da3e37-acc4-4717-a24b-f8979f425820 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:51:06.425098 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:06.425098 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:06.425098 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:06.425098 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:06.425098 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:06.425098 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:06.425098 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:06.425098 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:06.425098 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:06.425098 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:06.425098 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:06.425098 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:06.425098 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:06.425098 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:06.425098 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:06.426172 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:06.426172 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:06.426172 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:06.426172 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:06.426172 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:06.426172 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:06.426172 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:06.426172 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:06.426172 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:06.426172 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:06.426172 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:06.426172 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:06.426172 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:06.426172 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:06.426172 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:06.426172 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:06.427435 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:06.427435 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:06.427435 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:06.427435 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:06.427435 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:06.427435 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:06.427435 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:06.427435 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:06.427435 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:06.427435 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:06.427435 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:51:06.427435 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:06.427435 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 217/417] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:51:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:51:06.433652 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da3e37-acc4-4717-a24b-f8979f425820 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:06.553324 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da3e37-acc4-4717-a24b-f8979f425820 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:06.555075 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:06.590265 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1474294-20f7-4193-9261-34612c71ad71 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:06.601603 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:06.609611 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dcb7b7-cd97-443e-8797-85e4a5f71fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:06.610305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dcb7b7-cd97-443e-8797-85e4a5f71fbc None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:06.610979 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dcb7b7-cd97-443e-8797-85e4a5f71fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:06.611465 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dcb7b7-cd97-443e-8797-85e4a5f71fbc None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:06.612508 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dcb7b7-cd97-443e-8797-85e4a5f71fbc None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:06.613025 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dcb7b7-cd97-443e-8797-85e4a5f71fbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:06.614780 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dcb7b7-cd97-443e-8797-85e4a5f71fbc None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:06.624667 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 218/418] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 15:51:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:06.631510 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dcb7b7-cd97-443e-8797-85e4a5f71fbc None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:06.654002 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b726482-557f-48a8-ae4b-0190970e0d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:06.654902 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b726482-557f-48a8-ae4b-0190970e0d84 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:06.655187 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b726482-557f-48a8-ae4b-0190970e0d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:06.655466 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b726482-557f-48a8-ae4b-0190970e0d84 None admin] PATH_INFO: `/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d/groups/b7883f2649cd47f79cb62f0396bd0df7/roles/85dcd5509fca4911881d45baf5f1d5a5` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:06.658455 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b726482-557f-48a8-ae4b-0190970e0d84 None admin] RBAC: Authorizing `identity:create_grant(project_id=c9f5c8ab86d44f42af60e2d8442a4f3d, group_id=b7883f2649cd47f79cb62f0396bd0df7, role_id=85dcd5509fca4911881d45baf5f1d5a5)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:06.659919 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b726482-557f-48a8-ae4b-0190970e0d84 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:06.677069 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 201/419] 10.222.0.52 () {58 vars in 1442 bytes} [Thu Jul 21 15:51:06 2022] PUT /identity/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d/groups/b7883f2649cd47f79cb62f0396bd0df7/roles/85dcd5509fca4911881d45baf5f1d5a5 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 15:51:06.678495 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101233b-a5c0-40aa-8651-e1d5f090e103 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:06.678788 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101233b-a5c0-40aa-8651-e1d5f090e103 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:06.679117 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101233b-a5c0-40aa-8651-e1d5f090e103 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:06.680673 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 202/420] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:06.686691 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551853b2-cf7d-4b89-95c5-f674f340e76b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:06.687082 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551853b2-cf7d-4b89-95c5-f674f340e76b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:06.687266 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551853b2-cf7d-4b89-95c5-f674f340e76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:06.707787 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-551853b2-cf7d-4b89-95c5-f674f340e76b None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:06.707981 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-551853b2-cf7d-4b89-95c5-f674f340e76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:06.777772 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dcb7b7-cd97-443e-8797-85e4a5f71fbc None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:06.793159 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 203/421] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 15:51:06.819567 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551853b2-cf7d-4b89-95c5-f674f340e76b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:06.820518 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585defc7-c480-4f52-9506-2eda7df05e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:06.822465 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585defc7-c480-4f52-9506-2eda7df05e3b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:06.822754 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585defc7-c480-4f52-9506-2eda7df05e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:06.823034 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585defc7-c480-4f52-9506-2eda7df05e3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:06.898969 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:06.942623 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:06.956620 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585defc7-c480-4f52-9506-2eda7df05e3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:06.958624 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585defc7-c480-4f52-9506-2eda7df05e3b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:06.981262 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551853b2-cf7d-4b89-95c5-f674f340e76b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:07.022227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129ca6d-d430-4594-8ddd-4e89ff51b995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:07.023166 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129ca6d-d430-4594-8ddd-4e89ff51b995 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:07.023512 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129ca6d-d430-4594-8ddd-4e89ff51b995 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:07.023852 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129ca6d-d430-4594-8ddd-4e89ff51b995 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:07.035630 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5129ca6d-d430-4594-8ddd-4e89ff51b995 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:07.044006 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-5129ca6d-d430-4594-8ddd-4e89ff51b995 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:51:07.044006 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:07.044006 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:07.044006 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:07.044006 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:07.044006 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:07.044006 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:07.044006 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:07.044006 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:07.044006 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:07.044006 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:07.044006 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:07.044006 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:07.044006 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:07.044006 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:07.044006 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:07.044969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:07.044969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:07.044969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:07.044969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:07.044969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:07.044969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:07.044969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:07.044969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:07.044969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:07.044969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:07.044969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:07.044969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:07.044969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:07.044969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:07.044969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:07.044969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:07.045837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:07.045837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:07.045837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:07.045837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:07.045837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:07.045837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:07.045837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:07.045837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:07.045837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:07.045837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:07.045837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 15:51:07.045837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:07.045837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 204/422] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:51:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:51:07.052254 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129ca6d-d430-4594-8ddd-4e89ff51b995 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:07.105151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 219/423] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:51:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 15:51:07.113314 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585defc7-c480-4f52-9506-2eda7df05e3b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:07.169721 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129ca6d-d430-4594-8ddd-4e89ff51b995 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:07.192707 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:07.200963 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21295548-9310-4c8c-b73a-178248055923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:07.201779 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21295548-9310-4c8c-b73a-178248055923 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:07.202127 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21295548-9310-4c8c-b73a-178248055923 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:07.202618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21295548-9310-4c8c-b73a-178248055923 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:07.203772 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21295548-9310-4c8c-b73a-178248055923 None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:07.204911 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21295548-9310-4c8c-b73a-178248055923 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:07.207088 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21295548-9310-4c8c-b73a-178248055923 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:07.219592 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 205/424] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 15:51:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:51:07.221198 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:07.223704 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21295548-9310-4c8c-b73a-178248055923 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:07.297264 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585defc7-c480-4f52-9506-2eda7df05e3b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:07.339782 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21295548-9310-4c8c-b73a-178248055923 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:07.373856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cfc5bb-78b2-4f45-bfc7-7ce6790a9dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:07.375327 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfc5bb-78b2-4f45-bfc7-7ce6790a9dca None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:07.377867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfc5bb-78b2-4f45-bfc7-7ce6790a9dca None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:07.378350 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfc5bb-78b2-4f45-bfc7-7ce6790a9dca None admin] PATH_INFO: `/v3/projects/8edda125d8bb41498eca65801bd10882` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:07.380989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cfc5bb-78b2-4f45-bfc7-7ce6790a9dca None admin] RBAC: Authorizing `identity:get_project(project_id=8edda125d8bb41498eca65801bd10882)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:07.386093 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cfc5bb-78b2-4f45-bfc7-7ce6790a9dca None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:07.388180 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 220/425] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:51:07 2022] GET /identity/v3/projects/8edda125d8bb41498eca65801bd10882 => generated 298 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:07.394198 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cfc5bb-78b2-4f45-bfc7-7ce6790a9dca None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:07.421389 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e4248-eb94-463d-b453-0591fdae9443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:07.426090 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e4248-eb94-463d-b453-0591fdae9443 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:07.426640 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e4248-eb94-463d-b453-0591fdae9443 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:07.427116 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e4248-eb94-463d-b453-0591fdae9443 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:07.497116 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:07.510840 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4e4248-eb94-463d-b453-0591fdae9443 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:07.512570 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4e4248-eb94-463d-b453-0591fdae9443 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:07.569391 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cfc5bb-78b2-4f45-bfc7-7ce6790a9dca None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:07.606049 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a0340-bb6d-4a5b-85fe-5bc044684708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:07.607197 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a0340-bb6d-4a5b-85fe-5bc044684708 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:07.607628 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a0340-bb6d-4a5b-85fe-5bc044684708 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:07.608124 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a0340-bb6d-4a5b-85fe-5bc044684708 None admin] PATH_INFO: `/v3/groups/b7883f2649cd47f79cb62f0396bd0df7` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:07.609676 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a0340-bb6d-4a5b-85fe-5bc044684708 None admin] RBAC: Authorizing `identity:get_group(group_id=b7883f2649cd47f79cb62f0396bd0df7)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:07.611918 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a0340-bb6d-4a5b-85fe-5bc044684708 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:07.614134 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 221/426] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 21 15:51:07 2022] GET /identity/v3/groups/b7883f2649cd47f79cb62f0396bd0df7 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:07.614443 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:07.618577 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58ddd8-a445-4d78-9b69-bf8744356185 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:07.619144 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58ddd8-a445-4d78-9b69-bf8744356185 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:07.619711 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58ddd8-a445-4d78-9b69-bf8744356185 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:07.625758 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 206/428] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:51:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 15:51:07.625758 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e4248-eb94-463d-b453-0591fdae9443 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:07.626206 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 222/428] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:07.628066 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3fa0d4-b931-4de0-854b-2c5f9527fb4c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:07.628556 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3fa0d4-b931-4de0-854b-2c5f9527fb4c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:07.629001 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3fa0d4-b931-4de0-854b-2c5f9527fb4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:07.648249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2a3fa0d4-b931-4de0-854b-2c5f9527fb4c None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:07.650708 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2a3fa0d4-b931-4de0-854b-2c5f9527fb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:07.727293 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 223/429] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:51:07.731502 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:07.733412 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3fa0d4-b931-4de0-854b-2c5f9527fb4c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:07.742785 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e4248-eb94-463d-b453-0591fdae9443 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:07.780555 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3473b5-ad81-46b0-9908-4ebafb98ea7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:07.781432 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3473b5-ad81-46b0-9908-4ebafb98ea7c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:07.781897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3473b5-ad81-46b0-9908-4ebafb98ea7c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:07.782313 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3473b5-ad81-46b0-9908-4ebafb98ea7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:07.783339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3473b5-ad81-46b0-9908-4ebafb98ea7c None admin] RBAC: Adding query filter params (group.id=b7883f2649cd47f79cb62f0396bd0df7, role.id=54195e660fc349978b39d5794765a095, scope.project.id=8edda125d8bb41498eca65801bd10882) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:07.783865 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3473b5-ad81-46b0-9908-4ebafb98ea7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:07.785650 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3473b5-ad81-46b0-9908-4ebafb98ea7c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:07.792848 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 207/430] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jul 21 15:51:07 2022] GET /identity/v3/role_assignments?group.id=b7883f2649cd47f79cb62f0396bd0df7&scope.project.id=8edda125d8bb41498eca65801bd10882&role.id=54195e660fc349978b39d5794765a095 => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:07.802433 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3473b5-ad81-46b0-9908-4ebafb98ea7c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:07.925365 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3fa0d4-b931-4de0-854b-2c5f9527fb4c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:07.973950 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:07.983041 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42409c-5a13-4543-b204-01542d8d40e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:07.983923 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42409c-5a13-4543-b204-01542d8d40e2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:07.984211 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42409c-5a13-4543-b204-01542d8d40e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:07.984487 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42409c-5a13-4543-b204-01542d8d40e2 None admin] PATH_INFO: `/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:07.986288 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a42409c-5a13-4543-b204-01542d8d40e2 None admin] RBAC: Authorizing `identity:get_project(project_id=c9f5c8ab86d44f42af60e2d8442a4f3d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:07.987761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a42409c-5a13-4543-b204-01542d8d40e2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:07.989502 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 224/431] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:51:07 2022] GET /identity/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:07.996355 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42409c-5a13-4543-b204-01542d8d40e2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:08.019937 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3473b5-ad81-46b0-9908-4ebafb98ea7c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:08.067390 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4befb6f-4639-48f1-8d56-37b7c543ab8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:08.068537 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4befb6f-4639-48f1-8d56-37b7c543ab8b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:08.069040 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4befb6f-4639-48f1-8d56-37b7c543ab8b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:08.069517 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4befb6f-4639-48f1-8d56-37b7c543ab8b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:08.082138 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4befb6f-4639-48f1-8d56-37b7c543ab8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:08.090818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b4befb6f-4639-48f1-8d56-37b7c543ab8b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 15:51:08.090818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:08.090818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:08.090818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:08.090818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:08.090818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:08.090818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:08.090818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:08.090818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:08.090818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:08.090818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:08.090818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:08.090818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:08.090818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:08.090818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:08.090818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:08.091764 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:08.091764 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:08.091764 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:08.091764 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:08.091764 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:08.091764 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:08.091764 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:08.091764 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:08.091764 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:08.091764 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:08.091764 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:08.091764 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:08.091764 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:08.091764 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:08.091764 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:08.091764 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:08.092597 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:08.092597 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:08.092597 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:08.092597 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:08.092597 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:08.092597 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:08.092597 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:08.092597 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:08.092597 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:08.092597 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:08.092597 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 15:51:08.092597 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:08.095890 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 208/432] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 15:51:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:08.102196 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4befb6f-4639-48f1-8d56-37b7c543ab8b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:08.128799 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:08.149740 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42409c-5a13-4543-b204-01542d8d40e2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:08.203989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044cb50-1f7c-47c9-aae0-0f066a44eeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:08.205301 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044cb50-1f7c-47c9-aae0-0f066a44eeae None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:08.205814 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044cb50-1f7c-47c9-aae0-0f066a44eeae None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:08.206522 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044cb50-1f7c-47c9-aae0-0f066a44eeae None admin] PATH_INFO: `/v3/groups/b7883f2649cd47f79cb62f0396bd0df7` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:08.208503 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8044cb50-1f7c-47c9-aae0-0f066a44eeae None admin] RBAC: Authorizing `identity:get_group(group_id=b7883f2649cd47f79cb62f0396bd0df7)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:08.210812 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8044cb50-1f7c-47c9-aae0-0f066a44eeae None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:08.214021 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 225/433] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 21 15:51:07 2022] GET /identity/v3/groups/b7883f2649cd47f79cb62f0396bd0df7 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:08.226130 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044cb50-1f7c-47c9-aae0-0f066a44eeae None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:08.256618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4befb6f-4639-48f1-8d56-37b7c543ab8b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:08.302243 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e539f48-4c76-4481-ad31-3bd66225913d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:08.302929 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e539f48-4c76-4481-ad31-3bd66225913d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:08.303220 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e539f48-4c76-4481-ad31-3bd66225913d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:08.303525 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e539f48-4c76-4481-ad31-3bd66225913d None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:08.304892 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e539f48-4c76-4481-ad31-3bd66225913d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:08.304892 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e539f48-4c76-4481-ad31-3bd66225913d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:08.308427 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e539f48-4c76-4481-ad31-3bd66225913d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:08.317728 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 209/434] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:51:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:08.323740 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e539f48-4c76-4481-ad31-3bd66225913d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:08.332547 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:08.375964 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044cb50-1f7c-47c9-aae0-0f066a44eeae None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:08.417375 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c4feb-20d3-4926-8f58-8a303ffff618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:08.418475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c4feb-20d3-4926-8f58-8a303ffff618 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:08.418989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c4feb-20d3-4926-8f58-8a303ffff618 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:08.419465 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c4feb-20d3-4926-8f58-8a303ffff618 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:08.420520 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c4feb-20d3-4926-8f58-8a303ffff618 None admin] RBAC: Adding query filter params (group.id=b7883f2649cd47f79cb62f0396bd0df7, role.id=54195e660fc349978b39d5794765a095, scope.project.id=c9f5c8ab86d44f42af60e2d8442a4f3d) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:08.421069 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c4feb-20d3-4926-8f58-8a303ffff618 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:08.422833 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c4feb-20d3-4926-8f58-8a303ffff618 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:08.430964 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:08.431793 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 226/435] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jul 21 15:51:08 2022] GET /identity/v3/role_assignments?group.id=b7883f2649cd47f79cb62f0396bd0df7&scope.project.id=c9f5c8ab86d44f42af60e2d8442a4f3d&role.id=54195e660fc349978b39d5794765a095 => generated 611 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:51:08.460853 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e539f48-4c76-4481-ad31-3bd66225913d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:08.502652 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c196b9a-262a-4fd4-b57c-94c03de774f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:08.503377 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c196b9a-262a-4fd4-b57c-94c03de774f1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:08.503667 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c196b9a-262a-4fd4-b57c-94c03de774f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:08.503954 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c196b9a-262a-4fd4-b57c-94c03de774f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:08.594434 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c196b9a-262a-4fd4-b57c-94c03de774f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:08.596010 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c196b9a-262a-4fd4-b57c-94c03de774f1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:08.683688 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:08.713938 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 210/436] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:51:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 15:51:08.717438 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c4feb-20d3-4926-8f58-8a303ffff618 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:08.858288 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c4feb-20d3-4926-8f58-8a303ffff618 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:08.892999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:08.899765 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd56164-5f7b-442c-a9f5-649f583ac491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:08.900839 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd56164-5f7b-442c-a9f5-649f583ac491 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:08.901247 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd56164-5f7b-442c-a9f5-649f583ac491 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:08.901695 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd56164-5f7b-442c-a9f5-649f583ac491 None admin] PATH_INFO: `/v3/projects/8edda125d8bb41498eca65801bd10882` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:08.903019 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd56164-5f7b-442c-a9f5-649f583ac491 None admin] RBAC: Authorizing `identity:get_project(project_id=8edda125d8bb41498eca65801bd10882)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:08.904525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd56164-5f7b-442c-a9f5-649f583ac491 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:08.906922 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 227/437] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:51:08 2022] GET /identity/v3/projects/8edda125d8bb41498eca65801bd10882 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:51:08.912937 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c196b9a-262a-4fd4-b57c-94c03de774f1 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:09.020703 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c196b9a-262a-4fd4-b57c-94c03de774f1 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:09.022989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db3af9-4249-4475-a591-2002376bc2c0 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:09.023279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db3af9-4249-4475-a591-2002376bc2c0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:09.023545 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db3af9-4249-4475-a591-2002376bc2c0 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:09.025085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 228/438] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:09.036159 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8aec48-c2ae-4b65-8e07-3e87ac87ec2c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:09.036466 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8aec48-c2ae-4b65-8e07-3e87ac87ec2c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:09.036719 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8aec48-c2ae-4b65-8e07-3e87ac87ec2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:09.058523 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd3709-e5d4-4c2a-b95f-c2003b1d44ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:09.059392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd3709-e5d4-4c2a-b95f-c2003b1d44ba None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:09.059677 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd3709-e5d4-4c2a-b95f-c2003b1d44ba None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:09.059959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd3709-e5d4-4c2a-b95f-c2003b1d44ba None admin] PATH_INFO: `/v3/groups/b7883f2649cd47f79cb62f0396bd0df7` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:09.060888 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-cd8aec48-c2ae-4b65-8e07-3e87ac87ec2c None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:09.061162 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cd3709-e5d4-4c2a-b95f-c2003b1d44ba None admin] RBAC: Authorizing `identity:get_group(group_id=b7883f2649cd47f79cb62f0396bd0df7)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:09.062238 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cd8aec48-c2ae-4b65-8e07-3e87ac87ec2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:09.062668 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cd3709-e5d4-4c2a-b95f-c2003b1d44ba None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:09.065162 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 211/439] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 21 15:51:08 2022] GET /identity/v3/groups/b7883f2649cd47f79cb62f0396bd0df7 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:09.070633 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd3709-e5d4-4c2a-b95f-c2003b1d44ba None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:09.137987 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:09.145138 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 229/440] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 15:51:09.169521 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8aec48-c2ae-4b65-8e07-3e87ac87ec2c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:09.203071 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd3709-e5d4-4c2a-b95f-c2003b1d44ba None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:09.256212 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b891905-c369-42b6-8311-2d02e48f3c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:09.258469 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b891905-c369-42b6-8311-2d02e48f3c69 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:09.258469 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b891905-c369-42b6-8311-2d02e48f3c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:09.258469 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b891905-c369-42b6-8311-2d02e48f3c69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:09.268771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b891905-c369-42b6-8311-2d02e48f3c69 None admin] RBAC: Adding query filter params (group.id=b7883f2649cd47f79cb62f0396bd0df7, role.id=85dcd5509fca4911881d45baf5f1d5a5, scope.project.id=8edda125d8bb41498eca65801bd10882) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:09.268771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b891905-c369-42b6-8311-2d02e48f3c69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:09.268771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b891905-c369-42b6-8311-2d02e48f3c69 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:09.268771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 212/441] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jul 21 15:51:09 2022] GET /identity/v3/role_assignments?group.id=b7883f2649cd47f79cb62f0396bd0df7&scope.project.id=8edda125d8bb41498eca65801bd10882&role.id=85dcd5509fca4911881d45baf5f1d5a5 => generated 611 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:51:09.268771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7aecc5-045d-419a-a22d-1a9b3013ab4b None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:09.269476 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7aecc5-045d-419a-a22d-1a9b3013ab4b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:09.269476 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7aecc5-045d-419a-a22d-1a9b3013ab4b None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:09.269881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 213/442] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:09.276012 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cdbcd9-e1ee-4763-94c7-7243ffe74f65 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:09.276332 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cdbcd9-e1ee-4763-94c7-7243ffe74f65 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:09.276577 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cdbcd9-e1ee-4763-94c7-7243ffe74f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:09.300849 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-16cdbcd9-e1ee-4763-94c7-7243ffe74f65 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:09.302152 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-16cdbcd9-e1ee-4763-94c7-7243ffe74f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:09.316137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8aec48-c2ae-4b65-8e07-3e87ac87ec2c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:09.355532 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:09.367809 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa9dc7-6de9-40ed-9ca9-962a465dca18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:09.368721 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa9dc7-6de9-40ed-9ca9-962a465dca18 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:09.369012 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa9dc7-6de9-40ed-9ca9-962a465dca18 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:09.369283 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa9dc7-6de9-40ed-9ca9-962a465dca18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:09.382481 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa9dc7-6de9-40ed-9ca9-962a465dca18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:09.390603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e9fa9dc7-6de9-40ed-9ca9-962a465dca18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:51:09.390603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:09.390603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:09.390603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:09.390603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:09.390603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:09.390603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:09.390603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:09.390603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:09.390603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:09.390603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:09.390603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:09.390603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:09.390603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:09.390603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:09.390603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:09.397382 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:09.397382 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:09.397382 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:09.397382 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:09.397382 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:09.397382 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:09.397382 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:09.397382 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:09.397382 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:09.397382 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:09.397382 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:09.397382 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:09.397382 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:09.397382 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:09.397382 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:09.397382 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:09.398423 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:09.398423 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:09.398423 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:09.398423 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:09.398423 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:09.398423 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:09.398423 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:09.398423 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:09.398423 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:09.398423 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:09.398423 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:51:09.398423 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:09.398423 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 230/443] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:51:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:51:09.399113 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 214/444] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 15:51:09.399186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa9dc7-6de9-40ed-9ca9-962a465dca18 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:09.419949 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cdbcd9-e1ee-4763-94c7-7243ffe74f65 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:09.479091 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:09.532983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa9dc7-6de9-40ed-9ca9-962a465dca18 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:09.583691 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686f4e26-883c-42cb-aff7-c8be8b379918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:09.583897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cdbcd9-e1ee-4763-94c7-7243ffe74f65 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:09.584374 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f4e26-883c-42cb-aff7-c8be8b379918 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:09.584658 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f4e26-883c-42cb-aff7-c8be8b379918 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:09.584935 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f4e26-883c-42cb-aff7-c8be8b379918 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:09.585747 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686f4e26-883c-42cb-aff7-c8be8b379918 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:09.586062 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686f4e26-883c-42cb-aff7-c8be8b379918 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:09.589650 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686f4e26-883c-42cb-aff7-c8be8b379918 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:09.601831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 231/445] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:51:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:51:09.607374 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686f4e26-883c-42cb-aff7-c8be8b379918 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:09.628198 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784af607-190b-41b8-b49a-d43dde4f7814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:09.630064 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784af607-190b-41b8-b49a-d43dde4f7814 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:09.630064 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784af607-190b-41b8-b49a-d43dde4f7814 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:09.630064 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784af607-190b-41b8-b49a-d43dde4f7814 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:09.642735 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784af607-190b-41b8-b49a-d43dde4f7814 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:09.651227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-784af607-190b-41b8-b49a-d43dde4f7814 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 15:51:09.651227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:09.651227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:09.651227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:09.651227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:09.651227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:09.651227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:09.651227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:09.651227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:09.651227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:09.651227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:09.651227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:09.651227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:09.651227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:09.651227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:09.651227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:09.652182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:09.652182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:09.652182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:09.652182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:09.652182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:09.652182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:09.652182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:09.652182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:09.652182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:09.652182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:09.652182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:09.652182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:09.652182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:09.652182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:09.652182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:09.652182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:09.653025 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:09.653025 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:09.653025 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:09.653025 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:09.653025 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:09.653025 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:09.653025 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:09.653025 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:09.653025 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:09.653025 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:09.653025 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 15:51:09.653025 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:09.653686 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 215/446] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 15:51:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:51:09.659204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784af607-190b-41b8-b49a-d43dde4f7814 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:09.678583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:09.755818 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686f4e26-883c-42cb-aff7-c8be8b379918 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:09.795712 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:09.804622 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784af607-190b-41b8-b49a-d43dde4f7814 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:09.805556 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f38843-4ff3-4914-b888-ef83fcd1c9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:09.806238 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f38843-4ff3-4914-b888-ef83fcd1c9f5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:09.806523 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f38843-4ff3-4914-b888-ef83fcd1c9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:09.806804 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f38843-4ff3-4914-b888-ef83fcd1c9f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:09.855718 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce715eb6-727d-429f-98e2-91037004ba36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:09.856509 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce715eb6-727d-429f-98e2-91037004ba36 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:09.856798 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce715eb6-727d-429f-98e2-91037004ba36 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:09.857080 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce715eb6-727d-429f-98e2-91037004ba36 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:09.857893 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce715eb6-727d-429f-98e2-91037004ba36 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:09.858207 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce715eb6-727d-429f-98e2-91037004ba36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:09.859647 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce715eb6-727d-429f-98e2-91037004ba36 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:09.870255 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 216/447] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:51:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:51:09.875923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce715eb6-727d-429f-98e2-91037004ba36 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:09.888713 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f38843-4ff3-4914-b888-ef83fcd1c9f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:09.890245 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f38843-4ff3-4914-b888-ef83fcd1c9f5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:09.953096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:09.984819 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce715eb6-727d-429f-98e2-91037004ba36 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:10.011508 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 232/448] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:51:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 15:51:10.017824 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f38843-4ff3-4914-b888-ef83fcd1c9f5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:10.023067 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad02c8-7ab5-4934-b8a2-f4bd32f2bd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:10.023730 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad02c8-7ab5-4934-b8a2-f4bd32f2bd48 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:10.024017 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad02c8-7ab5-4934-b8a2-f4bd32f2bd48 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:10.024298 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad02c8-7ab5-4934-b8a2-f4bd32f2bd48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:10.057900 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:10.101513 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ad02c8-7ab5-4934-b8a2-f4bd32f2bd48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:10.103077 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ad02c8-7ab5-4934-b8a2-f4bd32f2bd48 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:10.127941 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f38843-4ff3-4914-b888-ef83fcd1c9f5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:10.164435 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5fc261-4e90-4b03-b111-b26ff9c008d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:10.165298 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fc261-4e90-4b03-b111-b26ff9c008d1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:10.165672 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fc261-4e90-4b03-b111-b26ff9c008d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:10.165980 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fc261-4e90-4b03-b111-b26ff9c008d1 None admin] PATH_INFO: `/v3/projects/f2cb644b2d994629b4b82065db94ebac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:10.167207 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5fc261-4e90-4b03-b111-b26ff9c008d1 None admin] RBAC: Authorizing `identity:get_project(project_id=f2cb644b2d994629b4b82065db94ebac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:10.168675 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5fc261-4e90-4b03-b111-b26ff9c008d1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:10.170602 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 233/449] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:51:10 2022] GET /identity/v3/projects/f2cb644b2d994629b4b82065db94ebac => generated 344 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:51:10.176118 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5fc261-4e90-4b03-b111-b26ff9c008d1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:10.198950 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:10.207748 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 217/450] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:51:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 15:51:10.213200 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad02c8-7ab5-4934-b8a2-f4bd32f2bd48 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:10.290559 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:10.302796 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5fc261-4e90-4b03-b111-b26ff9c008d1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:10.336402 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad02c8-7ab5-4934-b8a2-f4bd32f2bd48 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:10.342541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765fdece-b2fb-4524-bbd1-714c95515edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:10.343913 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765fdece-b2fb-4524-bbd1-714c95515edc None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:10.344502 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765fdece-b2fb-4524-bbd1-714c95515edc None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:10.344971 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765fdece-b2fb-4524-bbd1-714c95515edc None admin] PATH_INFO: `/v3/groups/8e23812fddc84ecfa4ea331a431b0e91` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:10.346971 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765fdece-b2fb-4524-bbd1-714c95515edc None admin] RBAC: Authorizing `identity:get_group(group_id=8e23812fddc84ecfa4ea331a431b0e91)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:10.349752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765fdece-b2fb-4524-bbd1-714c95515edc None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:10.352373 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 234/451] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 21 15:51:10 2022] GET /identity/v3/groups/8e23812fddc84ecfa4ea331a431b0e91 => generated 229 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:51:10.360931 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765fdece-b2fb-4524-bbd1-714c95515edc None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:10.373413 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7d8f1-6f45-4237-af4f-1dab3aa8e6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:10.374311 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7d8f1-6f45-4237-af4f-1dab3aa8e6ab None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:10.374594 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7d8f1-6f45-4237-af4f-1dab3aa8e6ab None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:10.374870 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7d8f1-6f45-4237-af4f-1dab3aa8e6ab None admin] PATH_INFO: `/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:10.376760 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a7d8f1-6f45-4237-af4f-1dab3aa8e6ab None admin] RBAC: Authorizing `identity:get_project(project_id=c9f5c8ab86d44f42af60e2d8442a4f3d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:10.378338 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a7d8f1-6f45-4237-af4f-1dab3aa8e6ab None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:10.380456 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 218/452] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:51:10 2022] GET /identity/v3/projects/c9f5c8ab86d44f42af60e2d8442a4f3d => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:51:10.385846 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7d8f1-6f45-4237-af4f-1dab3aa8e6ab None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:10.456199 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:10.472797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765fdece-b2fb-4524-bbd1-714c95515edc None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:10.491138 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7d8f1-6f45-4237-af4f-1dab3aa8e6ab None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:10.507766 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f664c91-b239-41d9-91e1-bcaf47394a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:10.508422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f664c91-b239-41d9-91e1-bcaf47394a25 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:10.508702 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f664c91-b239-41d9-91e1-bcaf47394a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:10.508978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f664c91-b239-41d9-91e1-bcaf47394a25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:10.509846 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f664c91-b239-41d9-91e1-bcaf47394a25 None admin] RBAC: Adding query filter params (group.id=8e23812fddc84ecfa4ea331a431b0e91, role.id=475521b8a0174037b162f92022aea1a1, scope.project.id=f2cb644b2d994629b4b82065db94ebac) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:10.510158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f664c91-b239-41d9-91e1-bcaf47394a25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:10.511624 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f664c91-b239-41d9-91e1-bcaf47394a25 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:10.518667 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 235/453] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jul 21 15:51:10 2022] GET /identity/v3/role_assignments?group.id=8e23812fddc84ecfa4ea331a431b0e91&scope.project.id=f2cb644b2d994629b4b82065db94ebac&role.id=475521b8a0174037b162f92022aea1a1 => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:51:10.520545 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:10.529455 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec0b8d-8860-4eef-bdd4-04282c60822e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:10.530482 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec0b8d-8860-4eef-bdd4-04282c60822e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:10.530838 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec0b8d-8860-4eef-bdd4-04282c60822e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:10.531210 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec0b8d-8860-4eef-bdd4-04282c60822e None admin] PATH_INFO: `/v3/groups/b7883f2649cd47f79cb62f0396bd0df7` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:10.532669 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec0b8d-8860-4eef-bdd4-04282c60822e None admin] RBAC: Authorizing `identity:get_group(group_id=b7883f2649cd47f79cb62f0396bd0df7)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:10.534277 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec0b8d-8860-4eef-bdd4-04282c60822e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:10.536160 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 219/454] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 21 15:51:10 2022] GET /identity/v3/groups/b7883f2649cd47f79cb62f0396bd0df7 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:51:10.544732 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f664c91-b239-41d9-91e1-bcaf47394a25 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:10.673597 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f664c91-b239-41d9-91e1-bcaf47394a25 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:10.712659 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b393006-d6c9-4bc9-ab50-c98d7280d2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:10.713321 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b393006-d6c9-4bc9-ab50-c98d7280d2ac None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:10.713633 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b393006-d6c9-4bc9-ab50-c98d7280d2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:10.713912 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b393006-d6c9-4bc9-ab50-c98d7280d2ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:10.714755 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b393006-d6c9-4bc9-ab50-c98d7280d2ac None admin] RBAC: Adding query filter params (group.id=b7883f2649cd47f79cb62f0396bd0df7, role.id=85dcd5509fca4911881d45baf5f1d5a5, scope.project.id=c9f5c8ab86d44f42af60e2d8442a4f3d) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:10.715071 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b393006-d6c9-4bc9-ab50-c98d7280d2ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:10.716555 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b393006-d6c9-4bc9-ab50-c98d7280d2ac None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:10.723479 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 236/455] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jul 21 15:51:10 2022] GET /identity/v3/role_assignments?group.id=b7883f2649cd47f79cb62f0396bd0df7&scope.project.id=c9f5c8ab86d44f42af60e2d8442a4f3d&role.id=85dcd5509fca4911881d45baf5f1d5a5 => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:13.342411 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987c87f-8d9f-4fef-8716-08e768f8c7da None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:13.343893 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987c87f-8d9f-4fef-8716-08e768f8c7da None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:13.344618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987c87f-8d9f-4fef-8716-08e768f8c7da None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:13.348610 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 220/456] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:13.354619 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07595538-f778-4dd6-b4b5-6c2546f56ee0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:13.356019 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07595538-f778-4dd6-b4b5-6c2546f56ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:13.356755 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07595538-f778-4dd6-b4b5-6c2546f56ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:13.388673 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-07595538-f778-4dd6-b4b5-6c2546f56ee0 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:13.390876 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-07595538-f778-4dd6-b4b5-6c2546f56ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:13.411711 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:13.482275 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 237/457] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:51:13.518446 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0987c87f-8d9f-4fef-8716-08e768f8c7da None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:13.658380 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0987c87f-8d9f-4fef-8716-08e768f8c7da None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:13.711229 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30559f-22b4-436c-ba6f-d01a87115e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:13.712300 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30559f-22b4-436c-ba6f-d01a87115e06 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:13.712765 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30559f-22b4-436c-ba6f-d01a87115e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:13.713192 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30559f-22b4-436c-ba6f-d01a87115e06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:13.725955 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea30559f-22b4-436c-ba6f-d01a87115e06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:13.727674 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea30559f-22b4-436c-ba6f-d01a87115e06 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:13.751392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ea30559f-22b4-436c-ba6f-d01a87115e06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:13.751392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:13.751392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:13.751392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:13.751392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:13.751392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:13.751392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:13.751392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:13.751392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:13.751392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:13.751392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:13.751392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:13.751392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:13.751392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:13.751392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:13.752572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:13.752572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:13.752572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:13.752572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:13.752572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:13.752572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:13.752572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:13.752572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:13.752572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:13.752572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:13.752572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:13.752572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:13.752572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:13.752572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:13.752572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:13.752572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:13.752572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:13.753734 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:13.753734 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:13.753734 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:13.753734 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:13.760364 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 221/458] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:13.761781 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07595538-f778-4dd6-b4b5-6c2546f56ee0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:13.832074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a550e8b4-f82c-4d4e-9a1e-cb2ed111c3ad None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:13.832629 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a550e8b4-f82c-4d4e-9a1e-cb2ed111c3ad None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:13.833158 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a550e8b4-f82c-4d4e-9a1e-cb2ed111c3ad None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:13.834755 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 222/459] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 15:51:13.854179 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb9de5-3d60-4e7d-9929-ceff4167cfe0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:13.854702 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb9de5-3d60-4e7d-9929-ceff4167cfe0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:13.855132 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb9de5-3d60-4e7d-9929-ceff4167cfe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:13.883648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1ccb9de5-3d60-4e7d-9929-ceff4167cfe0 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:13.884876 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1ccb9de5-3d60-4e7d-9929-ceff4167cfe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:13.936665 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07595538-f778-4dd6-b4b5-6c2546f56ee0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:13.959651 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:13.979439 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160178d-4377-4840-83c0-83792b6ea877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:13.980125 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160178d-4377-4840-83c0-83792b6ea877 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:13.980406 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160178d-4377-4840-83c0-83792b6ea877 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:13.980678 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160178d-4377-4840-83c0-83792b6ea877 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:13.981414 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160178d-4377-4840-83c0-83792b6ea877 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:13.981859 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160178d-4377-4840-83c0-83792b6ea877 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:13.983372 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160178d-4377-4840-83c0-83792b6ea877 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:13.991591 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 223/460] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 15:51:13.993082 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be844996-1b5a-48f7-8145-ffc8fe2a4aaa None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:13.993404 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be844996-1b5a-48f7-8145-ffc8fe2a4aaa None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:13.993716 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be844996-1b5a-48f7-8145-ffc8fe2a4aaa None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:13.995332 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 224/461] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:14.000955 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb601ad-1392-4d13-8776-c46417db8ee7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:14.001292 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb601ad-1392-4d13-8776-c46417db8ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:14.001606 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb601ad-1392-4d13-8776-c46417db8ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:14.028704 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 238/462] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:51:14.034509 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160178d-4377-4840-83c0-83792b6ea877 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:14.035118 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-acb601ad-1392-4d13-8776-c46417db8ee7 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:14.036557 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-acb601ad-1392-4d13-8776-c46417db8ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:14.045286 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:14.138581 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 225/463] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 15:51:14.139945 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb601ad-1392-4d13-8776-c46417db8ee7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:14.177464 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160178d-4377-4840-83c0-83792b6ea877 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:14.214343 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc6386-1fa4-4ee5-b49d-805a18fe42d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:14.215148 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc6386-1fa4-4ee5-b49d-805a18fe42d4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:14.215450 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc6386-1fa4-4ee5-b49d-805a18fe42d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:14.215743 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc6386-1fa4-4ee5-b49d-805a18fe42d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:14.228127 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc6386-1fa4-4ee5-b49d-805a18fe42d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:14.229896 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc6386-1fa4-4ee5-b49d-805a18fe42d4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:14.253074 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-86dc6386-1fa4-4ee5-b49d-805a18fe42d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:14.253074 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:14.253074 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:14.253074 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:14.253074 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:14.253074 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:14.253074 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:14.253074 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:14.253074 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:14.253074 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:14.253074 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:14.253074 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:14.253074 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:14.253074 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:14.253074 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:14.254055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:14.254055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:14.254055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:14.254055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:14.254055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:14.254055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:14.254055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:14.254055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:14.254055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:14.254055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:14.254055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:14.254055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:14.254055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:14.254055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:14.254055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:14.254055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:14.254055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:14.254912 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:14.254912 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:14.254912 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:14.254912 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:14.254912 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 239/464] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:51:14.260002 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc6386-1fa4-4ee5-b49d-805a18fe42d4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:14.260254 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb601ad-1392-4d13-8776-c46417db8ee7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:14.272252 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:14.302069 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a3494c-485c-448d-92d9-c1b239af5db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:14.303175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3494c-485c-448d-92d9-c1b239af5db1 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:14.303655 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3494c-485c-448d-92d9-c1b239af5db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:14.304171 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3494c-485c-448d-92d9-c1b239af5db1 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:14.305089 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a3494c-485c-448d-92d9-c1b239af5db1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:14.306851 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a3494c-485c-448d-92d9-c1b239af5db1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:14.327655 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:14.337727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 226/465] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 15:51:14 2022] POST /identity/v3/users => generated 246 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:51:14.337727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a3494c-485c-448d-92d9-c1b239af5db1 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:14.407251 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc6386-1fa4-4ee5-b49d-805a18fe42d4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:14.465079 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6250e10-f1da-4f38-9f61-7cb0745434df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:14.466215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6250e10-f1da-4f38-9f61-7cb0745434df None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:14.466698 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6250e10-f1da-4f38-9f61-7cb0745434df None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:14.467147 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6250e10-f1da-4f38-9f61-7cb0745434df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:14.481422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6250e10-f1da-4f38-9f61-7cb0745434df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:14.483231 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6250e10-f1da-4f38-9f61-7cb0745434df None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:14.485702 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a3494c-485c-448d-92d9-c1b239af5db1 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:14.504215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-f6250e10-f1da-4f38-9f61-7cb0745434df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:14.504215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:14.504215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:14.504215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:14.504215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:14.504215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:14.504215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:14.504215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:14.504215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:14.504215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:14.504215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:14.504215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:14.504215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:14.504215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:14.504215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:14.505708 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:14.505708 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:14.505708 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:14.505708 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:14.505708 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:14.505708 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:14.505708 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:14.505708 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:14.505708 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:14.505708 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:14.505708 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:14.505708 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:14.505708 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:14.505708 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:14.505708 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:14.505708 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:14.505708 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:14.506856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:14.506856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:14.506856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:14.506856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:14.513539 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 240/466] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:14.515321 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d7d57-25bb-4ccc-94c6-01b47371d278 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:14.515770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d7d57-25bb-4ccc-94c6-01b47371d278 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:14.516221 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d7d57-25bb-4ccc-94c6-01b47371d278 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:14.518625 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 241/467] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:14.519565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d7d57-25bb-4ccc-94c6-01b47371d278 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:14.529031 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68beede0-7b9c-4fc4-9520-d37fe4d13290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:14.529986 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68beede0-7b9c-4fc4-9520-d37fe4d13290 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:14.530282 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68beede0-7b9c-4fc4-9520-d37fe4d13290 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:14.530561 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68beede0-7b9c-4fc4-9520-d37fe4d13290 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:14.532770 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68beede0-7b9c-4fc4-9520-d37fe4d13290 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:14.533100 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68beede0-7b9c-4fc4-9520-d37fe4d13290 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:14.535402 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68beede0-7b9c-4fc4-9520-d37fe4d13290 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:14.552329 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 227/468] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:14.554435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5bf477-7ba5-4ffd-b0d6-68bcdc5a2bd7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:14.554435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5bf477-7ba5-4ffd-b0d6-68bcdc5a2bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:14.554435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5bf477-7ba5-4ffd-b0d6-68bcdc5a2bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:14.572165 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9b5bf477-7ba5-4ffd-b0d6-68bcdc5a2bd7 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:14.573379 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9b5bf477-7ba5-4ffd-b0d6-68bcdc5a2bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:14.596652 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:14.671638 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d7d57-25bb-4ccc-94c6-01b47371d278 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:14.683963 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:14.687079 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 228/469] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:51:14.688313 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5bf477-7ba5-4ffd-b0d6-68bcdc5a2bd7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:14.739591 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33391db5-b0cd-40df-8dc3-74c365f53c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:14.740434 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33391db5-b0cd-40df-8dc3-74c365f53c50 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:14.740818 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33391db5-b0cd-40df-8dc3-74c365f53c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:14.741197 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33391db5-b0cd-40df-8dc3-74c365f53c50 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:14.742488 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33391db5-b0cd-40df-8dc3-74c365f53c50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:14.742488 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33391db5-b0cd-40df-8dc3-74c365f53c50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:14.744063 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33391db5-b0cd-40df-8dc3-74c365f53c50 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:14.757904 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 242/470] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:14.759089 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33391db5-b0cd-40df-8dc3-74c365f53c50 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:14.822308 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5bf477-7ba5-4ffd-b0d6-68bcdc5a2bd7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:14.872299 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca206c2-4744-4dce-9a0e-a5b27d060ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:14.873296 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca206c2-4744-4dce-9a0e-a5b27d060ffa None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:14.873809 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca206c2-4744-4dce-9a0e-a5b27d060ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:14.874247 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca206c2-4744-4dce-9a0e-a5b27d060ffa None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:14.875156 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca206c2-4744-4dce-9a0e-a5b27d060ffa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:14.877424 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca206c2-4744-4dce-9a0e-a5b27d060ffa None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:14.905021 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 229/471] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 15:51:14 2022] POST /identity/v3/users => generated 248 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:51:14.906174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca206c2-4744-4dce-9a0e-a5b27d060ffa None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:14.927852 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33391db5-b0cd-40df-8dc3-74c365f53c50 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:14.944074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:15.002287 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7960e991-3d58-42a5-9f3c-8938e9c80248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:15.003393 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960e991-3d58-42a5-9f3c-8938e9c80248 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:15.003790 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960e991-3d58-42a5-9f3c-8938e9c80248 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:15.004441 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960e991-3d58-42a5-9f3c-8938e9c80248 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:15.016940 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7960e991-3d58-42a5-9f3c-8938e9c80248 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:15.018860 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7960e991-3d58-42a5-9f3c-8938e9c80248 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:15.044564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-7960e991-3d58-42a5-9f3c-8938e9c80248 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:15.044564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:15.044564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:15.044564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:15.044564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:15.044564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:15.044564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:15.044564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:15.044564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:15.044564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:15.044564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:15.044564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:15.044564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:15.044564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:15.044564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:15.045764 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:15.045764 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:15.045764 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:15.045764 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:15.045764 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:15.045764 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:15.045764 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:15.045764 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:15.045764 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:15.045764 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:15.045764 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:15.045764 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:15.045764 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:15.045764 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:15.045764 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:15.045764 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:15.045764 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:15.046881 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:15.046881 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:15.046881 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:15.046881 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:15.048806 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 243/472] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:15.051458 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745fa7f-613b-49c5-b6bf-5723697247a0 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:15.052100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745fa7f-613b-49c5-b6bf-5723697247a0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:15.053184 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745fa7f-613b-49c5-b6bf-5723697247a0 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:15.060100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 244/473] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:15.061615 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f745fa7f-613b-49c5-b6bf-5723697247a0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:15.069105 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca206c2-4744-4dce-9a0e-a5b27d060ffa None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:15.094406 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:15.114917 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e91d1c-2eb1-4164-915d-febf45e4d787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:15.115603 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e91d1c-2eb1-4164-915d-febf45e4d787 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:15.115893 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e91d1c-2eb1-4164-915d-febf45e4d787 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:15.116179 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e91d1c-2eb1-4164-915d-febf45e4d787 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:15.116901 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e91d1c-2eb1-4164-915d-febf45e4d787 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:15.119748 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e91d1c-2eb1-4164-915d-febf45e4d787 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:15.146027 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 230/474] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 15:51:14 2022] POST /identity/v3/users => generated 248 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:51:15.147507 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d16a1-13fd-4200-9519-482f8d7a0c1c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:15.148467 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d16a1-13fd-4200-9519-482f8d7a0c1c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:15.148795 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d16a1-13fd-4200-9519-482f8d7a0c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:15.169471 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c83d16a1-13fd-4200-9519-482f8d7a0c1c None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:15.170881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c83d16a1-13fd-4200-9519-482f8d7a0c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:15.219950 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f745fa7f-613b-49c5-b6bf-5723697247a0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:15.268935 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92efd0d7-bf0e-40e0-97ae-49133c5e35c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:15.269652 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92efd0d7-bf0e-40e0-97ae-49133c5e35c6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:15.269945 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92efd0d7-bf0e-40e0-97ae-49133c5e35c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:15.270225 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92efd0d7-bf0e-40e0-97ae-49133c5e35c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:15.270978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92efd0d7-bf0e-40e0-97ae-49133c5e35c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:15.271297 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92efd0d7-bf0e-40e0-97ae-49133c5e35c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:15.272784 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92efd0d7-bf0e-40e0-97ae-49133c5e35c6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:15.274882 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:15.287171 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 245/475] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:15.288055 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 231/476] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:51:15.292687 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92efd0d7-bf0e-40e0-97ae-49133c5e35c6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:15.357314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d16a1-13fd-4200-9519-482f8d7a0c1c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:15.400219 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:15.425081 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92efd0d7-bf0e-40e0-97ae-49133c5e35c6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:15.471422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f7820-6b06-42ba-9523-8301d3e29996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:15.472133 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f7820-6b06-42ba-9523-8301d3e29996 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:15.472483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f7820-6b06-42ba-9523-8301d3e29996 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:15.473086 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f7820-6b06-42ba-9523-8301d3e29996 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:15.473870 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144f7820-6b06-42ba-9523-8301d3e29996 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:15.475347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144f7820-6b06-42ba-9523-8301d3e29996 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:15.499630 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d16a1-13fd-4200-9519-482f8d7a0c1c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:15.505771 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 246/477] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 15:51:15 2022] POST /identity/v3/users => generated 249 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:51:15.549332 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4cb254-a57a-47bd-94b6-2ad8e41bec91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:15.550151 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4cb254-a57a-47bd-94b6-2ad8e41bec91 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:15.550434 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4cb254-a57a-47bd-94b6-2ad8e41bec91 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:15.550759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4cb254-a57a-47bd-94b6-2ad8e41bec91 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:15.551627 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4cb254-a57a-47bd-94b6-2ad8e41bec91 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:15.553137 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4cb254-a57a-47bd-94b6-2ad8e41bec91 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:15.561251 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.sql.core [None req-7c4cb254-a57a-47bd-94b6-2ad8e41bec91 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 15:51:15.561251 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 21 15:51:15.561251 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [parameters: {'id': '7af4baca55154101a90c9acd1e6d6666', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 21 15:51:15.561251 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80238) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 15:51:15.579591 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-7c4cb254-a57a-47bd-94b6-2ad8e41bec91 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 21 15:51:15.579591 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:15.579591 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 15:51:15.579591 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 15:51:15.579591 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 15:51:15.579591 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 15:51:15.579591 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 15:51:15.579591 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 15:51:15.579591 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 15:51:15.579591 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application conn.query(q) Jul 21 15:51:15.579591 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 15:51:15.579591 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 15:51:15.579591 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 15:51:15.579591 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application result.read() Jul 21 15:51:15.579591 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 15:51:15.582862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 15:51:15.582862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 15:51:15.582862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 15:51:15.582862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 15:51:15.582862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 15:51:15.582862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 15:51:15.582862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 15:51:15.582862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 15:51:15.582862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:15.582862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 15:51:15.582862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:15.582862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:15.582862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 15:51:15.582862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 15:51:15.582862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 21 15:51:15.582862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return ref.to_dict() Jul 21 15:51:15.582862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 15:51:15.582862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application next(self.gen) Jul 21 15:51:15.586223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 15:51:15.586223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application yield resource Jul 21 15:51:15.586223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 15:51:15.586223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application next(self.gen) Jul 21 15:51:15.586223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 15:51:15.586223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 15:51:15.586223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 15:51:15.586223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 15:51:15.586223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 15:51:15.586223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise self.value Jul 21 15:51:15.586223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 15:51:15.586223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 15:51:15.586223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 15:51:15.586223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application session.commit() Jul 21 15:51:15.586223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 15:51:15.586223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 15:51:15.586223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 15:51:15.588735 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 15:51:15.588735 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 15:51:15.588735 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self.session.flush() Jul 21 15:51:15.588735 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 15:51:15.588735 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 15:51:15.588735 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 15:51:15.588735 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 15:51:15.588735 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 15:51:15.588735 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application compat.raise_( Jul 21 15:51:15.588735 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 15:51:15.588735 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception Jul 21 15:51:15.588735 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 15:51:15.588735 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 15:51:15.588735 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 15:51:15.588735 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 15:51:15.588735 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 15:51:15.588735 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 15:51:15.591923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 15:51:15.591923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 15:51:15.591923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 21 15:51:15.591923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 21 15:51:15.591923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 15:51:15.591923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 15:51:15.591923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 15:51:15.591923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 15:51:15.591923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 15:51:15.591923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 15:51:15.591923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 15:51:15.591923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 15:51:15.591923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 15:51:15.591923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 15:51:15.591923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 15:51:15.591923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception Jul 21 15:51:15.593932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 15:51:15.593932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 15:51:15.593932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 15:51:15.593932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 15:51:15.593932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 15:51:15.593932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 15:51:15.593932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 15:51:15.593932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application conn.query(q) Jul 21 15:51:15.593932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 15:51:15.593932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 15:51:15.593932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 15:51:15.593932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application result.read() Jul 21 15:51:15.593932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 15:51:15.593932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 15:51:15.593932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 15:51:15.593932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 15:51:15.598878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 15:51:15.598878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 15:51:15.598878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 15:51:15.598878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 15:51:15.598878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 15:51:15.598878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 21 15:51:15.598878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [parameters: {'id': '7af4baca55154101a90c9acd1e6d6666', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 21 15:51:15.598878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 15:51:15.598878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:15.598878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 15:51:15.598878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:15.598878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:15.598878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:15.598878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:15.598878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:15.598878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:15.600840 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:15.600840 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:15.600840 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:15.600840 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:15.600840 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:15.600840 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:15.600840 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 21 15:51:15.600840 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 21 15:51:15.600840 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:15.600840 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:15.600840 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 21 15:51:15.600840 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 21 15:51:15.600840 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 15:51:15.600840 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 15:51:15.600840 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 21 15:51:15.600840 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:15.602813 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 232/478] 10.222.0.52 () {60 vars in 1131 bytes} [Thu Jul 21 15:51:15 2022] POST /identity/v3/roles => generated 144 bytes in 224 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:15.602947 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f7820-6b06-42ba-9523-8301d3e29996 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:15.733549 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f7820-6b06-42ba-9523-8301d3e29996 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:15.779650 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00350cc8-9015-4383-b3de-1ccdbd72af43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:15.780499 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00350cc8-9015-4383-b3de-1ccdbd72af43 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:15.780790 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00350cc8-9015-4383-b3de-1ccdbd72af43 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:15.781071 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00350cc8-9015-4383-b3de-1ccdbd72af43 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:15.786171 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:15.802025 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00350cc8-9015-4383-b3de-1ccdbd72af43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:15.812536 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-00350cc8-9015-4383-b3de-1ccdbd72af43 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 15:51:15.812536 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:15.812536 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:15.812536 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:15.812536 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:15.812536 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:15.812536 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:15.812536 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:15.812536 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:15.812536 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:15.812536 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:15.812536 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:15.812536 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:15.812536 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:15.812536 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:15.812536 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:15.813542 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:15.813542 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:15.813542 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:15.813542 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:15.813542 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:15.813542 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:15.813542 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:15.813542 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:15.813542 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:15.813542 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:15.813542 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:15.813542 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:15.813542 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:15.813542 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:15.813542 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:15.813542 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:15.814399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:15.814399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:15.814399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:15.814399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:15.814399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:15.814399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:15.814399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:15.814399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:15.814399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:15.814399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:15.814399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 15:51:15.814399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:15.814399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 247/479] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 15:51:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:15.822763 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4cb254-a57a-47bd-94b6-2ad8e41bec91 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:15.919713 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:15.975991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4cb254-a57a-47bd-94b6-2ad8e41bec91 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:16.033247 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36729de-edc2-44f9-a2be-920f5d67b5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:16.033940 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36729de-edc2-44f9-a2be-920f5d67b5a3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:16.034228 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36729de-edc2-44f9-a2be-920f5d67b5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:16.034510 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36729de-edc2-44f9-a2be-920f5d67b5a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:16.035328 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36729de-edc2-44f9-a2be-920f5d67b5a3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:16.035647 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36729de-edc2-44f9-a2be-920f5d67b5a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:16.037051 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36729de-edc2-44f9-a2be-920f5d67b5a3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:16.046584 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 233/480] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:51:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:16.902004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a11592-80f0-42fb-902b-187e6078ba5e None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:16.902566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a11592-80f0-42fb-902b-187e6078ba5e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:16.902566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a11592-80f0-42fb-902b-187e6078ba5e None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:16.904243 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 248/481] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:16.909934 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f211d-6f70-471a-b784-a22e73753295 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:16.910498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f211d-6f70-471a-b784-a22e73753295 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:16.910498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f211d-6f70-471a-b784-a22e73753295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:16.928981 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b40f211d-6f70-471a-b784-a22e73753295 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:16.930303 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b40f211d-6f70-471a-b784-a22e73753295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:17.019280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 234/482] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:51:17.072739 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a11592-80f0-42fb-902b-187e6078ba5e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:17.240278 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a11592-80f0-42fb-902b-187e6078ba5e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:17.294607 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf722d-0ad5-4c0d-b093-3d8abaa24805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:17.295430 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf722d-0ad5-4c0d-b093-3d8abaa24805 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:17.295719 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf722d-0ad5-4c0d-b093-3d8abaa24805 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:17.295999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf722d-0ad5-4c0d-b093-3d8abaa24805 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:17.314466 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbf722d-0ad5-4c0d-b093-3d8abaa24805 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:17.322283 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-7dbf722d-0ad5-4c0d-b093-3d8abaa24805 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:17.322283 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:17.322283 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:17.322283 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:17.322283 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:17.322283 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:17.322283 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:17.322283 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:17.322283 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:17.322283 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:17.322283 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:17.322283 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:17.322283 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:17.322283 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:17.322283 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:17.322283 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:17.323249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:17.323249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:17.323249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:17.323249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:17.323249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:17.323249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:17.323249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:17.323249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:17.323249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:17.323249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:17.323249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:17.323249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:17.323249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:17.323249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:17.323249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:17.323249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:17.324091 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:17.324091 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:17.324091 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:17.324091 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:17.324091 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:17.324091 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:17.324091 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:17.324091 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:17.324091 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:17.324091 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:17.324091 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:17.324091 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:17.324091 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 249/483] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 15:51:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:17.334759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f211d-6f70-471a-b784-a22e73753295 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:17.430090 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:17.504541 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f211d-6f70-471a-b784-a22e73753295 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:17.562751 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee501544-3282-466c-9cba-f63d7ba3ad20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:17.563807 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee501544-3282-466c-9cba-f63d7ba3ad20 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:17.564132 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee501544-3282-466c-9cba-f63d7ba3ad20 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:17.564415 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee501544-3282-466c-9cba-f63d7ba3ad20 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:17.565209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee501544-3282-466c-9cba-f63d7ba3ad20 None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:17.565528 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee501544-3282-466c-9cba-f63d7ba3ad20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:17.567048 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee501544-3282-466c-9cba-f63d7ba3ad20 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:17.576688 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 235/484] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 15:51:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:17.582438 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf722d-0ad5-4c0d-b093-3d8abaa24805 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:17.671812 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:17.736533 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf722d-0ad5-4c0d-b093-3d8abaa24805 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:17.772859 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b262e-b2a1-4586-8bbc-fe953320f18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:17.773592 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b262e-b2a1-4586-8bbc-fe953320f18f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:17.773875 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b262e-b2a1-4586-8bbc-fe953320f18f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:17.774152 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b262e-b2a1-4586-8bbc-fe953320f18f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:17.786329 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877b262e-b2a1-4586-8bbc-fe953320f18f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:17.787816 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877b262e-b2a1-4586-8bbc-fe953320f18f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:17.805657 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-877b262e-b2a1-4586-8bbc-fe953320f18f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:17.805657 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:17.805657 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:17.805657 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:17.805657 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:17.805657 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:17.805657 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:17.805657 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:17.805657 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:17.805657 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:17.805657 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:17.805657 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:17.805657 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:17.805657 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:17.805657 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:17.806590 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:17.806590 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:17.806590 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:17.806590 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:17.806590 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:17.806590 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:17.806590 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:17.806590 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:17.806590 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:17.806590 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:17.806590 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:17.806590 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:17.806590 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:17.806590 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:17.806590 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:17.806590 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:17.806590 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:17.807720 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:17.807720 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:17.807720 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:17.807720 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:17.807720 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 250/485] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:17.815766 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee501544-3282-466c-9cba-f63d7ba3ad20 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:17.998753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc80db-c0d7-4684-9410-8faf81d5b587 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:17.999282 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc80db-c0d7-4684-9410-8faf81d5b587 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:17.999569 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc80db-c0d7-4684-9410-8faf81d5b587 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:18.001986 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 251/486] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:18.008466 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609a58f-0fea-444c-a69e-d0a9780d1bc7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:18.008758 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609a58f-0fea-444c-a69e-d0a9780d1bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:18.009016 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609a58f-0fea-444c-a69e-d0a9780d1bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:18.012491 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee501544-3282-466c-9cba-f63d7ba3ad20 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:18.028215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e609a58f-0fea-444c-a69e-d0a9780d1bc7 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:18.029346 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e609a58f-0fea-444c-a69e-d0a9780d1bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:18.047044 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:18.097666 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1a655-f18c-407a-9393-579f8eb75a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:18.099055 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1a655-f18c-407a-9393-579f8eb75a56 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:18.099572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1a655-f18c-407a-9393-579f8eb75a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:18.100080 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1a655-f18c-407a-9393-579f8eb75a56 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:18.101490 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e1a655-f18c-407a-9393-579f8eb75a56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:18.101865 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e1a655-f18c-407a-9393-579f8eb75a56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:18.104348 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e1a655-f18c-407a-9393-579f8eb75a56 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:18.123866 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 236/487] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:18.131192 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1a655-f18c-407a-9393-579f8eb75a56 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:18.160071 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 252/488] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 15:51:18.161864 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87320749-e449-44a3-988d-412b8c1884f1 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:18.162273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87320749-e449-44a3-988d-412b8c1884f1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:18.162652 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87320749-e449-44a3-988d-412b8c1884f1 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:18.164465 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 253/489] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:18.176867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14884b29-7301-4e21-b2d9-f51df695a5a4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:18.177893 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14884b29-7301-4e21-b2d9-f51df695a5a4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:18.178178 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14884b29-7301-4e21-b2d9-f51df695a5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:18.201434 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-14884b29-7301-4e21-b2d9-f51df695a5a4 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:18.202758 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-14884b29-7301-4e21-b2d9-f51df695a5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:18.240223 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:18.284964 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 254/490] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 15:51:18.288752 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1a655-f18c-407a-9393-579f8eb75a56 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:18.289872 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14884b29-7301-4e21-b2d9-f51df695a5a4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:18.328786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b2aab-d2a1-4187-a6a7-066ace58db8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:18.330160 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b2aab-d2a1-4187-a6a7-066ace58db8b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:18.330477 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b2aab-d2a1-4187-a6a7-066ace58db8b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:18.330719 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b2aab-d2a1-4187-a6a7-066ace58db8b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:18.338662 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149b2aab-d2a1-4187-a6a7-066ace58db8b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:18.340130 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149b2aab-d2a1-4187-a6a7-066ace58db8b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:18.358184 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-149b2aab-d2a1-4187-a6a7-066ace58db8b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 15:51:18.358184 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:18.358184 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:18.358184 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:18.358184 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:18.358184 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:18.358184 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:18.358184 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:18.358184 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:18.358184 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:18.358184 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:18.358184 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:18.358184 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:51:18.358184 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:51:18.358184 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:51:18.358184 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:51:18.359392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:18.359392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:18.359392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:51:18.359392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:18.359392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:51:18.359392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:18.359392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:18.359392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:18.359392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:18.359392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:18.359392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:18.359392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:18.359392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:18.359392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:18.359392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:18.359392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:18.359392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:18.359392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:18.360593 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:18.360593 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:18.360593 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:51:18.360593 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:51:18.360593 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:51:18.360593 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:51:18.360593 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:51:18.360593 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:51:18.360593 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 15:51:18.360593 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:18.360593 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 237/491] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jul 21 15:51:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:51:18.362099 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6511a5-285b-4a23-b5c1-9d2d3e114ed3 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:18.362596 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6511a5-285b-4a23-b5c1-9d2d3e114ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:18.363049 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6511a5-285b-4a23-b5c1-9d2d3e114ed3 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:18.368975 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 238/492] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:18.368975 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6511a5-285b-4a23-b5c1-9d2d3e114ed3 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:18.421308 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:18.424372 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14884b29-7301-4e21-b2d9-f51df695a5a4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:18.471200 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb34203-fb4a-4838-b240-c487d5b29460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:18.472021 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb34203-fb4a-4838-b240-c487d5b29460 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:18.472304 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb34203-fb4a-4838-b240-c487d5b29460 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:18.472581 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb34203-fb4a-4838-b240-c487d5b29460 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:18.485113 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb34203-fb4a-4838-b240-c487d5b29460 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:18.492986 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-ecb34203-fb4a-4838-b240-c487d5b29460 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:18.492986 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:18.492986 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:18.492986 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:18.492986 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:18.492986 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:18.492986 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:18.492986 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:18.492986 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:18.492986 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:18.492986 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:18.492986 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:18.492986 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:18.492986 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:18.492986 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:18.492986 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:18.494207 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:18.494207 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:18.494207 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:18.494207 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:18.494207 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:18.494207 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:18.494207 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:18.494207 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:18.494207 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:18.494207 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:18.494207 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:18.494207 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:18.494207 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:18.494207 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:18.494207 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:18.494207 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:18.495052 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:18.495052 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:18.495052 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:18.495052 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:18.495052 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:18.495052 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:18.495052 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:18.495052 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:18.495052 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:18.495052 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:18.495052 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:18.495052 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:18.495052 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 255/493] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 15:51:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:18.495052 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb34203-fb4a-4838-b240-c487d5b29460 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:18.507918 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6511a5-285b-4a23-b5c1-9d2d3e114ed3 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:18.529366 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:18.548954 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f9879f-f422-40f7-906a-3b93c1a3c8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:18.549674 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9879f-f422-40f7-906a-3b93c1a3c8ef None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:18.549958 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9879f-f422-40f7-906a-3b93c1a3c8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:18.550226 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9879f-f422-40f7-906a-3b93c1a3c8ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:18.563846 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f9879f-f422-40f7-906a-3b93c1a3c8ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:18.572217 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-61f9879f-f422-40f7-906a-3b93c1a3c8ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:18.572217 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:18.572217 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:18.572217 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:18.572217 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:18.572217 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:18.572217 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:18.572217 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:18.572217 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:18.572217 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:18.572217 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:18.572217 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:18.572217 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:18.572217 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:18.572217 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:18.572217 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:18.573157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:18.573157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:18.573157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:18.573157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:18.573157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:18.573157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:18.573157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:18.573157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:18.573157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:18.573157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:18.573157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:18.573157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:18.573157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:18.573157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:18.573157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:18.573157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:18.579771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:18.579771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:18.579771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:18.579771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:18.579771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:18.579771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:18.579771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:18.579771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:18.579771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:18.579771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:18.579771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:18.579771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:18.579771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 239/494] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 15:51:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:18.579771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d29f4-427b-47e1-a630-2b08ec3410e9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:18.580799 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d29f4-427b-47e1-a630-2b08ec3410e9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:18.580799 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d29f4-427b-47e1-a630-2b08ec3410e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:18.597326 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-df7d29f4-427b-47e1-a630-2b08ec3410e9 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:18.598901 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-df7d29f4-427b-47e1-a630-2b08ec3410e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:18.622908 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb34203-fb4a-4838-b240-c487d5b29460 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:18.664026 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5158fc3-ae0b-4830-92a0-78a718612c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:18.665162 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5158fc3-ae0b-4830-92a0-78a718612c2c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:18.665557 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5158fc3-ae0b-4830-92a0-78a718612c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:18.666042 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5158fc3-ae0b-4830-92a0-78a718612c2c None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:18.667377 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5158fc3-ae0b-4830-92a0-78a718612c2c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:18.667833 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5158fc3-ae0b-4830-92a0-78a718612c2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:18.669307 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5158fc3-ae0b-4830-92a0-78a718612c2c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:18.687694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 240/495] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:51:18.687694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d29f4-427b-47e1-a630-2b08ec3410e9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:18.711661 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:18.715608 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 256/496] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jul 21 15:51:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:18.716688 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5158fc3-ae0b-4830-92a0-78a718612c2c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:18.815504 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d29f4-427b-47e1-a630-2b08ec3410e9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:18.820459 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:18.837251 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5158fc3-ae0b-4830-92a0-78a718612c2c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:18.865037 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcebc193-f05c-4434-a6f5-c5f34703694d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:18.865749 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcebc193-f05c-4434-a6f5-c5f34703694d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:18.866034 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcebc193-f05c-4434-a6f5-c5f34703694d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:18.866313 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcebc193-f05c-4434-a6f5-c5f34703694d None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:18.867068 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcebc193-f05c-4434-a6f5-c5f34703694d None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:18.867377 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcebc193-f05c-4434-a6f5-c5f34703694d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:18.868778 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcebc193-f05c-4434-a6f5-c5f34703694d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:18.878097 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 241/497] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 15:51:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:18.882804 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcebc193-f05c-4434-a6f5-c5f34703694d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:18.892183 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca7fb9-57b6-4e74-abe8-77292b5a43c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:18.892858 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca7fb9-57b6-4e74-abe8-77292b5a43c8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:18.893482 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca7fb9-57b6-4e74-abe8-77292b5a43c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:18.893482 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca7fb9-57b6-4e74-abe8-77292b5a43c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:18.894197 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca7fb9-57b6-4e74-abe8-77292b5a43c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:18.894513 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca7fb9-57b6-4e74-abe8-77292b5a43c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:18.895903 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca7fb9-57b6-4e74-abe8-77292b5a43c8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:18.909841 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 257/498] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 15:51:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:18.909841 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca7fb9-57b6-4e74-abe8-77292b5a43c8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:18.980739 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:18.997286 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcebc193-f05c-4434-a6f5-c5f34703694d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:19.033786 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca7fb9-57b6-4e74-abe8-77292b5a43c8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:19.037444 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e12b3f-994f-4c2b-9ba7-b0b0c5307089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:19.038725 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e12b3f-994f-4c2b-9ba7-b0b0c5307089 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:19.039199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e12b3f-994f-4c2b-9ba7-b0b0c5307089 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:19.039656 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e12b3f-994f-4c2b-9ba7-b0b0c5307089 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:19.060613 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e12b3f-994f-4c2b-9ba7-b0b0c5307089 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:19.068715 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-32e12b3f-994f-4c2b-9ba7-b0b0c5307089 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:19.068715 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:19.068715 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:19.068715 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:19.068715 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:19.068715 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:19.068715 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:19.068715 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:19.068715 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:19.068715 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:19.068715 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:19.068715 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:19.068715 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:19.068715 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:19.068715 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:19.068715 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:19.069920 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:19.069920 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:19.069920 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:19.069920 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:19.069920 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:19.069920 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:19.069920 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:19.069920 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:19.069920 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:19.069920 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:19.069920 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:19.069920 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:19.069920 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:19.069920 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:19.069920 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:19.069920 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:19.071127 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:19.071127 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:19.071127 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:19.071127 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:19.071127 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:19.071127 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:19.071127 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:19.071127 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:19.071127 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:19.071127 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:19.071127 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:19.071127 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:19.071127 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 242/499] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 15:51:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:19.071127 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e12b3f-994f-4c2b-9ba7-b0b0c5307089 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:19.074917 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2170819-4bca-4787-a84f-5073cfdaf193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:19.075580 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2170819-4bca-4787-a84f-5073cfdaf193 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:19.075861 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2170819-4bca-4787-a84f-5073cfdaf193 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:19.076224 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2170819-4bca-4787-a84f-5073cfdaf193 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:19.095825 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:19.148267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2170819-4bca-4787-a84f-5073cfdaf193 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:19.149895 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2170819-4bca-4787-a84f-5073cfdaf193 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:19.192007 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e12b3f-994f-4c2b-9ba7-b0b0c5307089 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:19.229411 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1179dce7-e098-468c-947c-b3dcf6fb5782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:19.230240 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179dce7-e098-468c-947c-b3dcf6fb5782 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:19.230526 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179dce7-e098-468c-947c-b3dcf6fb5782 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:19.230809 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179dce7-e098-468c-947c-b3dcf6fb5782 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:19.242820 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1179dce7-e098-468c-947c-b3dcf6fb5782 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:19.244497 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1179dce7-e098-468c-947c-b3dcf6fb5782 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:19.252491 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:19.263405 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 258/500] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:51:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 15:51:19.264250 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2170819-4bca-4787-a84f-5073cfdaf193 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:19.264862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-1179dce7-e098-468c-947c-b3dcf6fb5782 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:19.264862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:19.264862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:19.264862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:19.264862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:19.264862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:19.264862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:19.264862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:19.264862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:19.264862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:19.264862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:19.264862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:19.264862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:19.264862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:19.264862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:19.265727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:19.265727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:19.265727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:19.265727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:19.265727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:19.265727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:19.265727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:19.265727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:19.265727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:19.265727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:19.265727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:19.265727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:19.265727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:19.265727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:19.265727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:19.265727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:19.265727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:19.267154 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:19.267154 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:19.267154 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:19.267154 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:19.267154 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 243/501] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:19.271102 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137fe874-e38d-41ec-b8a2-eff0dd8dbb78 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:19.271394 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137fe874-e38d-41ec-b8a2-eff0dd8dbb78 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:19.271682 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137fe874-e38d-41ec-b8a2-eff0dd8dbb78 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:19.273616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 244/502] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:19.274988 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137fe874-e38d-41ec-b8a2-eff0dd8dbb78 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:19.332287 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:19.385712 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2170819-4bca-4787-a84f-5073cfdaf193 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:19.388981 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137fe874-e38d-41ec-b8a2-eff0dd8dbb78 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:19.422801 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99bdd7c-9223-41e6-8d0f-33fac83d87f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:19.423602 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99bdd7c-9223-41e6-8d0f-33fac83d87f6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:19.423885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99bdd7c-9223-41e6-8d0f-33fac83d87f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:19.424159 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99bdd7c-9223-41e6-8d0f-33fac83d87f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:19.425746 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd8562-13e0-495f-a332-75ca9f42e554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:19.426412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd8562-13e0-495f-a332-75ca9f42e554 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:19.426698 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd8562-13e0-495f-a332-75ca9f42e554 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:19.426985 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd8562-13e0-495f-a332-75ca9f42e554 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:19.427745 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd8562-13e0-495f-a332-75ca9f42e554 None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:19.428054 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd8562-13e0-495f-a332-75ca9f42e554 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:19.429456 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd8562-13e0-495f-a332-75ca9f42e554 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:19.435693 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99bdd7c-9223-41e6-8d0f-33fac83d87f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:19.437456 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99bdd7c-9223-41e6-8d0f-33fac83d87f6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:19.438991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 245/503] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 15:51:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:19.439829 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd8562-13e0-495f-a332-75ca9f42e554 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:19.460818 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e99bdd7c-9223-41e6-8d0f-33fac83d87f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:19.460818 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:19.460818 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:19.460818 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:19.460818 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:19.460818 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:19.460818 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:19.460818 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:19.460818 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:19.460818 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:19.460818 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:19.460818 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:19.460818 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:19.460818 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:19.460818 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:19.461861 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:19.461861 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:19.461861 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:19.461861 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:19.461861 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:19.461861 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:19.461861 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:19.461861 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:19.461861 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:19.461861 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:19.461861 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:19.461861 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:19.461861 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:19.461861 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:19.461861 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:19.461861 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:19.461861 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:19.462753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:19.462753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:19.462753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:19.462753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:19.462753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 259/504] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:19.463109 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99bdd7c-9223-41e6-8d0f-33fac83d87f6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:19.511070 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:19.554940 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd8562-13e0-495f-a332-75ca9f42e554 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:19.576561 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:19.580852 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99bdd7c-9223-41e6-8d0f-33fac83d87f6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:19.591047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f25ed-b310-49b4-997e-6d02602320be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:19.591775 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f25ed-b310-49b4-997e-6d02602320be None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:19.592065 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f25ed-b310-49b4-997e-6d02602320be None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:19.592345 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f25ed-b310-49b4-997e-6d02602320be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:19.604603 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f25ed-b310-49b4-997e-6d02602320be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:19.606126 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f25ed-b310-49b4-997e-6d02602320be None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:19.623178 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce81da-a440-4628-b3fb-b406fcf15c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:19.625194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce81da-a440-4628-b3fb-b406fcf15c44 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:19.625194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce81da-a440-4628-b3fb-b406fcf15c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:19.625194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce81da-a440-4628-b3fb-b406fcf15c44 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:19.625442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ce81da-a440-4628-b3fb-b406fcf15c44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:19.625635 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ce81da-a440-4628-b3fb-b406fcf15c44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:19.627303 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-056f25ed-b310-49b4-997e-6d02602320be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:19.627303 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:19.627303 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:19.627303 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:19.627303 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:19.627303 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:19.627303 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:19.627303 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:19.627303 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:19.627303 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:19.627303 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:19.627303 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:19.627303 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:19.627303 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:19.627303 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:19.628286 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:19.628286 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:19.628286 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:19.628286 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:19.628286 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:19.628286 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:19.628286 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:19.628286 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:19.628286 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:19.628286 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:19.628286 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:19.628286 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:19.628286 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:19.628286 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:19.628286 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:19.628286 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:19.628286 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:19.629212 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:19.629212 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:19.629212 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:19.629212 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:19.629440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ce81da-a440-4628-b3fb-b406fcf15c44 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:19.629509 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 246/505] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:19.629928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061d459-5e87-4181-b032-9a1374367803 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:19.630223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061d459-5e87-4181-b032-9a1374367803 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:19.630508 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061d459-5e87-4181-b032-9a1374367803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:19.645541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 260/506] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:19.646484 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce81da-a440-4628-b3fb-b406fcf15c44 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:19.650346 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1061d459-5e87-4181-b032-9a1374367803 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:19.651696 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1061d459-5e87-4181-b032-9a1374367803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:19.729535 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 247/507] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:51:19.730418 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061d459-5e87-4181-b032-9a1374367803 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:19.765245 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce81da-a440-4628-b3fb-b406fcf15c44 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:19.782374 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:19.807753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b278a7c-292c-46ed-9048-fbceac18e4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:19.808545 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b278a7c-292c-46ed-9048-fbceac18e4ec None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:19.808829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b278a7c-292c-46ed-9048-fbceac18e4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:19.809103 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b278a7c-292c-46ed-9048-fbceac18e4ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:19.820426 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b278a7c-292c-46ed-9048-fbceac18e4ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:19.821992 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b278a7c-292c-46ed-9048-fbceac18e4ec None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:19.840831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-1b278a7c-292c-46ed-9048-fbceac18e4ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:19.840831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:19.840831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:19.840831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:19.840831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:19.840831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:19.840831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:19.840831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:19.840831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:19.840831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:19.840831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:19.840831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:19.840831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:19.840831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:19.840831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:19.841884 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:19.841884 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:19.841884 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:19.841884 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:19.841884 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:19.841884 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:19.841884 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:19.841884 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:19.841884 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:19.841884 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:19.841884 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:19.841884 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:19.841884 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:19.841884 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:19.841884 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:19.841884 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:19.841884 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:19.842762 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:19.842762 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:19.842762 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:19.842762 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:19.842762 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 261/508] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:19.843045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b278a7c-292c-46ed-9048-fbceac18e4ec None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:19.855581 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061d459-5e87-4181-b032-9a1374367803 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:19.860810 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:19.892436 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a9fdd-31c6-40d5-a365-12b3802be760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:19.893104 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a9fdd-31c6-40d5-a365-12b3802be760 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:19.893388 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a9fdd-31c6-40d5-a365-12b3802be760 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:19.893703 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a9fdd-31c6-40d5-a365-12b3802be760 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:19.894439 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2a9fdd-31c6-40d5-a365-12b3802be760 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:19.894752 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2a9fdd-31c6-40d5-a365-12b3802be760 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:19.896161 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2a9fdd-31c6-40d5-a365-12b3802be760 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:19.907839 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 248/509] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:19.908676 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a9fdd-31c6-40d5-a365-12b3802be760 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:19.966541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b278a7c-292c-46ed-9048-fbceac18e4ec None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:20.006805 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff18e8cd-7757-4813-9f1e-4eec5e34ede2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:20.007482 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18e8cd-7757-4813-9f1e-4eec5e34ede2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:20.007770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18e8cd-7757-4813-9f1e-4eec5e34ede2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:20.008051 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18e8cd-7757-4813-9f1e-4eec5e34ede2 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:20.008788 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff18e8cd-7757-4813-9f1e-4eec5e34ede2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:20.009102 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff18e8cd-7757-4813-9f1e-4eec5e34ede2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:20.010540 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff18e8cd-7757-4813-9f1e-4eec5e34ede2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:20.018207 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a9fdd-31c6-40d5-a365-12b3802be760 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:20.019784 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:20.022776 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 262/510] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:20.023629 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff18e8cd-7757-4813-9f1e-4eec5e34ede2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:20.061683 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d7fa3-7be8-4a19-8d22-d651fe005b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:20.062493 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d7fa3-7be8-4a19-8d22-d651fe005b34 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:20.062780 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d7fa3-7be8-4a19-8d22-d651fe005b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:20.063056 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d7fa3-7be8-4a19-8d22-d651fe005b34 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:20.071602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660d7fa3-7be8-4a19-8d22-d651fe005b34 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:20.073071 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660d7fa3-7be8-4a19-8d22-d651fe005b34 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:20.088314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-660d7fa3-7be8-4a19-8d22-d651fe005b34 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 15:51:20.088314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:20.088314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:20.088314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:20.088314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:20.088314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:20.088314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:20.088314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:20.088314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:20.088314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:20.088314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:20.088314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:20.088314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:51:20.088314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:51:20.088314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:51:20.088314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:51:20.089412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:20.089412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:20.089412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:51:20.089412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:20.089412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:51:20.089412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:20.089412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:20.089412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:20.089412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:20.089412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:20.089412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:20.089412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:20.089412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:20.089412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:20.089412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:20.089412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:20.089412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:20.089412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:20.090368 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:20.090368 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:20.090368 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:51:20.090368 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:51:20.090368 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:51:20.090368 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:51:20.090368 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:51:20.090368 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:51:20.090368 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 15:51:20.090368 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:20.090368 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 249/511] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:51:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:20.090368 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d7fa3-7be8-4a19-8d22-d651fe005b34 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:20.116607 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:20.150421 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff18e8cd-7757-4813-9f1e-4eec5e34ede2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:20.190267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe096b2-436b-42e4-992d-d8540f90975c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:20.191169 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe096b2-436b-42e4-992d-d8540f90975c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:20.191449 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe096b2-436b-42e4-992d-d8540f90975c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:20.191721 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe096b2-436b-42e4-992d-d8540f90975c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:20.202721 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe096b2-436b-42e4-992d-d8540f90975c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:20.204232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe096b2-436b-42e4-992d-d8540f90975c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:20.216280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d7fa3-7be8-4a19-8d22-d651fe005b34 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:20.223751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-cfe096b2-436b-42e4-992d-d8540f90975c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:20.223751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:20.223751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:20.223751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:20.223751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:20.223751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:20.223751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:20.223751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:20.223751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:20.223751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:20.223751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:20.223751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:20.223751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:20.223751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:20.223751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:20.224826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:20.224826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:20.224826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:20.224826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:20.224826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:20.224826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:20.224826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:20.224826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:20.224826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:20.224826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:20.224826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:20.224826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:20.224826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:20.224826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:20.224826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:20.224826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:20.224826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:20.230767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:20.230767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:20.230767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:20.230767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:20.230767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 263/512] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:20.230767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe096b2-436b-42e4-992d-d8540f90975c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:20.267969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783ba32-d393-4a92-b981-2cb39e82cc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:20.268641 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783ba32-d393-4a92-b981-2cb39e82cc6d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:20.268925 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783ba32-d393-4a92-b981-2cb39e82cc6d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:20.269205 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783ba32-d393-4a92-b981-2cb39e82cc6d None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:20.269980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9783ba32-d393-4a92-b981-2cb39e82cc6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:20.270292 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9783ba32-d393-4a92-b981-2cb39e82cc6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:20.271696 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9783ba32-d393-4a92-b981-2cb39e82cc6d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:20.284867 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 250/513] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:20.286605 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783ba32-d393-4a92-b981-2cb39e82cc6d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:20.318031 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:20.338224 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe096b2-436b-42e4-992d-d8540f90975c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:20.362799 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:20.375066 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a937a9-b7b9-41a7-b256-4ae560bae38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:20.375818 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a937a9-b7b9-41a7-b256-4ae560bae38f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:20.376105 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a937a9-b7b9-41a7-b256-4ae560bae38f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:20.376382 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a937a9-b7b9-41a7-b256-4ae560bae38f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:20.384723 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a937a9-b7b9-41a7-b256-4ae560bae38f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:20.386285 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a937a9-b7b9-41a7-b256-4ae560bae38f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:20.402615 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d3a937a9-b7b9-41a7-b256-4ae560bae38f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 15:51:20.402615 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:20.402615 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:20.402615 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:20.402615 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:20.402615 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:20.402615 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:20.402615 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:20.402615 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:20.402615 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:20.402615 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:20.402615 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:20.402615 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:51:20.402615 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:51:20.402615 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:51:20.402615 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:51:20.403830 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:20.403830 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:20.403830 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:51:20.403830 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:20.403830 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:51:20.403830 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:20.403830 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:20.403830 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:20.403830 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:20.403830 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:20.403830 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:20.403830 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:20.403830 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:20.403830 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:20.403830 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:20.403830 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:20.403830 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:20.403830 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:20.404989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:20.404989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:20.404989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:51:20.404989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:51:20.404989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:51:20.404989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:51:20.404989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:51:20.404989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:51:20.404989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 15:51:20.404989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:20.404989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 264/514] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:51:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:20.404989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a937a9-b7b9-41a7-b256-4ae560bae38f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:20.414160 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783ba32-d393-4a92-b981-2cb39e82cc6d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:20.455689 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae855b-d32e-434d-9f8e-47744d27434b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:20.456497 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae855b-d32e-434d-9f8e-47744d27434b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:20.456813 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae855b-d32e-434d-9f8e-47744d27434b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:20.457091 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae855b-d32e-434d-9f8e-47744d27434b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:20.467846 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae855b-d32e-434d-9f8e-47744d27434b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:20.469334 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae855b-d32e-434d-9f8e-47744d27434b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:20.486513 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-beae855b-d32e-434d-9f8e-47744d27434b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:20.486513 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:20.486513 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:20.486513 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:20.486513 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:20.486513 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:20.486513 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:20.486513 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:20.486513 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:20.486513 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:20.486513 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:20.486513 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:20.486513 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:51:20.486513 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:51:20.486513 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:51:20.487648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:51:20.487648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:20.487648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:20.487648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:20.487648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:20.487648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:20.487648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:20.487648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:20.487648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:20.487648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:20.487648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:20.487648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:20.487648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:20.487648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:20.487648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:20.487648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:20.487648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:20.488718 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:51:20.488718 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:51:20.488718 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:51:20.488718 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:51:20.488718 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:51:20.488718 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:51:20.488718 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:20.488718 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:20.489387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 251/515] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:51:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:20.490283 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae855b-d32e-434d-9f8e-47744d27434b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:20.513532 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a937a9-b7b9-41a7-b256-4ae560bae38f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:20.548872 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b119ae-84d1-4670-8184-40a443c78a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:20.549687 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b119ae-84d1-4670-8184-40a443c78a79 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:20.549968 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b119ae-84d1-4670-8184-40a443c78a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:20.550242 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b119ae-84d1-4670-8184-40a443c78a79 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:20.551087 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b119ae-84d1-4670-8184-40a443c78a79 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:20.551402 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b119ae-84d1-4670-8184-40a443c78a79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:20.552864 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b119ae-84d1-4670-8184-40a443c78a79 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:20.572869 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 265/516] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Jul 21 15:51:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:20.573760 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b119ae-84d1-4670-8184-40a443c78a79 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:20.581341 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:20.601702 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae855b-d32e-434d-9f8e-47744d27434b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:20.629741 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:20.653824 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598fccf9-1ab0-4b93-9861-658c8a36dfbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:20.654906 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598fccf9-1ab0-4b93-9861-658c8a36dfbe None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:20.655363 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598fccf9-1ab0-4b93-9861-658c8a36dfbe None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:20.655828 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598fccf9-1ab0-4b93-9861-658c8a36dfbe None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:20.657694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598fccf9-1ab0-4b93-9861-658c8a36dfbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:20.658190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598fccf9-1ab0-4b93-9861-658c8a36dfbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:20.660581 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598fccf9-1ab0-4b93-9861-658c8a36dfbe None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:20.683460 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 252/517] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:20.683460 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598fccf9-1ab0-4b93-9861-658c8a36dfbe None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:20.693530 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b119ae-84d1-4670-8184-40a443c78a79 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:20.730896 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f038503-0683-458e-a8c2-4745704262ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:20.731707 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f038503-0683-458e-a8c2-4745704262ae None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:20.731993 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f038503-0683-458e-a8c2-4745704262ae None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:20.732270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f038503-0683-458e-a8c2-4745704262ae None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:20.740909 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f038503-0683-458e-a8c2-4745704262ae None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:20.742443 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f038503-0683-458e-a8c2-4745704262ae None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:20.758027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-0f038503-0683-458e-a8c2-4745704262ae None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 15:51:20.758027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:20.758027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:20.758027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:20.758027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:20.758027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:20.758027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:20.758027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:20.758027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:20.758027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:20.758027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:20.758027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:20.758027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:51:20.758027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:51:20.758027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:51:20.758027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:51:20.759197 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:20.759197 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:20.759197 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:51:20.759197 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:20.759197 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:51:20.759197 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:20.759197 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:20.759197 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:20.759197 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:20.759197 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:20.759197 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:20.759197 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:20.759197 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:20.759197 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:20.759197 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:20.759197 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:20.759197 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:20.759197 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:20.760126 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:20.760126 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:20.760126 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:51:20.760126 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:51:20.760126 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:51:20.760126 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:51:20.760126 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:51:20.760126 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:51:20.760126 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 15:51:20.760126 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:20.760126 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 266/518] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 15:51:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:20.760126 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f038503-0683-458e-a8c2-4745704262ae None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:20.793523 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598fccf9-1ab0-4b93-9861-658c8a36dfbe None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:20.830491 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e9532-c687-494a-a701-d38615161ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:20.831490 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e9532-c687-494a-a701-d38615161ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:20.831922 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e9532-c687-494a-a701-d38615161ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:20.832354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e9532-c687-494a-a701-d38615161ae0 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:20.833727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e9532-c687-494a-a701-d38615161ae0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:20.834204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e9532-c687-494a-a701-d38615161ae0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:20.836599 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e9532-c687-494a-a701-d38615161ae0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:20.866443 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f038503-0683-458e-a8c2-4745704262ae None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:20.868007 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:20.885617 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:20.889485 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 253/519] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Jul 21 15:51:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:20.890464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e9532-c687-494a-a701-d38615161ae0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:20.910500 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9fb32-a21b-4dda-8240-bda269c5973d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:20.911247 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9fb32-a21b-4dda-8240-bda269c5973d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:20.911550 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9fb32-a21b-4dda-8240-bda269c5973d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:20.911831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9fb32-a21b-4dda-8240-bda269c5973d None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:20.912633 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9fb32-a21b-4dda-8240-bda269c5973d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:20.912950 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9fb32-a21b-4dda-8240-bda269c5973d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:20.914457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9fb32-a21b-4dda-8240-bda269c5973d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:20.932348 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 267/520] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:51:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:20.933160 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9fb32-a21b-4dda-8240-bda269c5973d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:20.998509 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e9532-c687-494a-a701-d38615161ae0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:21.033497 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b5d0f-0ea1-421b-a8ef-3ffaf4367f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:21.034201 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b5d0f-0ea1-421b-a8ef-3ffaf4367f81 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:21.034545 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b5d0f-0ea1-421b-a8ef-3ffaf4367f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:21.034827 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b5d0f-0ea1-421b-a8ef-3ffaf4367f81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:21.042085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9fb32-a21b-4dda-8240-bda269c5973d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:21.078025 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75b771-9a61-4a50-b0dd-5df64d300586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:21.078719 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75b771-9a61-4a50-b0dd-5df64d300586 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:21.079533 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75b771-9a61-4a50-b0dd-5df64d300586 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:21.079808 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75b771-9a61-4a50-b0dd-5df64d300586 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:21.080512 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75b771-9a61-4a50-b0dd-5df64d300586 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:21.081993 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75b771-9a61-4a50-b0dd-5df64d300586 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:21.102768 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 268/521] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 15:51:20 2022] POST /identity/v3/users => generated 247 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:51:21.103601 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75b771-9a61-4a50-b0dd-5df64d300586 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:21.111597 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3b5d0f-0ea1-421b-a8ef-3ffaf4367f81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:21.113145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3b5d0f-0ea1-421b-a8ef-3ffaf4367f81 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:21.121622 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:21.127581 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:21.257415 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 254/522] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:51:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 15:51:21.257415 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b5d0f-0ea1-421b-a8ef-3ffaf4367f81 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:21.266982 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75b771-9a61-4a50-b0dd-5df64d300586 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:21.321089 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad3d2f-c8b6-4250-b42e-bfe29fd57241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:21.321886 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad3d2f-c8b6-4250-b42e-bfe29fd57241 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:21.322181 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad3d2f-c8b6-4250-b42e-bfe29fd57241 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:21.322457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad3d2f-c8b6-4250-b42e-bfe29fd57241 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:21.323327 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad3d2f-c8b6-4250-b42e-bfe29fd57241 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:21.323636 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad3d2f-c8b6-4250-b42e-bfe29fd57241 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:21.325125 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad3d2f-c8b6-4250-b42e-bfe29fd57241 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:21.347305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 269/523] 10.222.0.52 () {56 vars in 1139 bytes} [Thu Jul 21 15:51:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:21.348349 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad3d2f-c8b6-4250-b42e-bfe29fd57241 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:21.386661 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:21.389870 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b5d0f-0ea1-421b-a8ef-3ffaf4367f81 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:21.433622 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1478ef-572e-4636-ad20-f88564e77365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:21.434291 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1478ef-572e-4636-ad20-f88564e77365 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:21.434577 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1478ef-572e-4636-ad20-f88564e77365 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:21.434861 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1478ef-572e-4636-ad20-f88564e77365 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:21.447376 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:21.477191 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad3d2f-c8b6-4250-b42e-bfe29fd57241 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:21.519163 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1daea6-317e-44a6-b265-f896dcbec89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:21.519940 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1daea6-317e-44a6-b265-f896dcbec89f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:21.520228 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1daea6-317e-44a6-b265-f896dcbec89f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:21.520515 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1daea6-317e-44a6-b265-f896dcbec89f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:21.521363 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1daea6-317e-44a6-b265-f896dcbec89f None admin] RBAC: Adding query filter params (role.id=64d94d0d5a7640818c21aa5a622624ab, scope.project.id=bb1ac00308714c39a79aae09fbb73021, user.id=092553a86f8848dba8c7271f5c0b2c46) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:21.521715 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1daea6-317e-44a6-b265-f896dcbec89f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:21.523202 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1daea6-317e-44a6-b265-f896dcbec89f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:21.524904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1478ef-572e-4636-ad20-f88564e77365 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:21.527794 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1478ef-572e-4636-ad20-f88564e77365 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:21.529767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 270/524] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:51:21 2022] GET /identity/v3/role_assignments?user.id=092553a86f8848dba8c7271f5c0b2c46&scope.project.id=bb1ac00308714c39a79aae09fbb73021&role.id=64d94d0d5a7640818c21aa5a622624ab => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:21.530610 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1daea6-317e-44a6-b265-f896dcbec89f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:21.680685 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1daea6-317e-44a6-b265-f896dcbec89f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:21.685967 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:21.695495 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 255/525] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:51:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 15:51:21.696577 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1478ef-572e-4636-ad20-f88564e77365 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:21.745690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada0c87-3429-49cf-9ed8-7abd11fbc6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:21.746502 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada0c87-3429-49cf-9ed8-7abd11fbc6a5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:21.746791 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada0c87-3429-49cf-9ed8-7abd11fbc6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:21.747073 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada0c87-3429-49cf-9ed8-7abd11fbc6a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:21.759955 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:21.762864 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bada0c87-3429-49cf-9ed8-7abd11fbc6a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:21.764363 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bada0c87-3429-49cf-9ed8-7abd11fbc6a5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:21.791979 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-bada0c87-3429-49cf-9ed8-7abd11fbc6a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:21.791979 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:21.791979 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:21.791979 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:21.791979 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:21.791979 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:21.791979 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:21.791979 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:21.791979 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:21.791979 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:21.791979 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:21.791979 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:21.791979 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:21.791979 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:21.791979 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:21.792978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:21.792978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:21.792978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:21.792978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:21.792978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:21.792978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:21.792978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:21.792978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:21.792978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:21.792978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:21.792978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:21.792978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:21.792978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:21.792978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:21.792978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:21.792978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:21.792978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:21.799963 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:21.799963 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:21.799963 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:21.799963 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:21.799963 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 271/526] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:21.801938 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada0c87-3429-49cf-9ed8-7abd11fbc6a5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:21.834986 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1478ef-572e-4636-ad20-f88564e77365 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:21.878461 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b430c68-3b7d-4b0e-9fb4-0c43792a215d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:21.879677 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b430c68-3b7d-4b0e-9fb4-0c43792a215d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:21.880195 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b430c68-3b7d-4b0e-9fb4-0c43792a215d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:21.880697 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b430c68-3b7d-4b0e-9fb4-0c43792a215d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:21.966178 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada0c87-3429-49cf-9ed8-7abd11fbc6a5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:21.971086 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:21.974190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b430c68-3b7d-4b0e-9fb4-0c43792a215d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:21.975920 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b430c68-3b7d-4b0e-9fb4-0c43792a215d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:22.019172 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8902532-6490-442b-8364-c631960c9c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:22.020569 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8902532-6490-442b-8364-c631960c9c6a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:22.021161 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8902532-6490-442b-8364-c631960c9c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:22.021806 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8902532-6490-442b-8364-c631960c9c6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:22.036654 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8902532-6490-442b-8364-c631960c9c6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:22.038762 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8902532-6490-442b-8364-c631960c9c6a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:22.065701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d8902532-6490-442b-8364-c631960c9c6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:22.065701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:22.065701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:22.065701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:22.065701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:22.065701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:22.065701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:22.065701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:22.065701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:22.065701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:22.065701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:22.065701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:22.065701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:22.065701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:22.065701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:22.066671 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:22.066671 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:22.066671 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:22.066671 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:22.066671 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:22.066671 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:22.066671 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:22.066671 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:22.066671 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:22.066671 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:22.066671 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:22.066671 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:22.066671 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:22.066671 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:22.066671 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:22.066671 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:22.066671 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:22.067641 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:22.067641 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:22.067641 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:22.067641 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:22.067641 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 272/527] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:22.069186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8902532-6490-442b-8364-c631960c9c6a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:22.089161 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 256/528] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:51:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 15:51:22.090636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b430c68-3b7d-4b0e-9fb4-0c43792a215d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:22.134989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:22.198220 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8902532-6490-442b-8364-c631960c9c6a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:22.205696 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:22.208772 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b430c68-3b7d-4b0e-9fb4-0c43792a215d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:22.248323 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26683407-f6a0-4be5-8b35-03a2966cc566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:22.249424 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26683407-f6a0-4be5-8b35-03a2966cc566 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:22.249981 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26683407-f6a0-4be5-8b35-03a2966cc566 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:22.250400 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26683407-f6a0-4be5-8b35-03a2966cc566 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:22.251484 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26683407-f6a0-4be5-8b35-03a2966cc566 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:22.251941 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26683407-f6a0-4be5-8b35-03a2966cc566 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:22.252068 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b5cfc-077a-46af-9f38-1361d915dfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:22.252840 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b5cfc-077a-46af-9f38-1361d915dfca None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:22.253223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b5cfc-077a-46af-9f38-1361d915dfca None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:22.253630 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b5cfc-077a-46af-9f38-1361d915dfca None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:22.254286 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26683407-f6a0-4be5-8b35-03a2966cc566 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:22.254569 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201b5cfc-077a-46af-9f38-1361d915dfca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:22.254978 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201b5cfc-077a-46af-9f38-1361d915dfca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:22.256566 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201b5cfc-077a-46af-9f38-1361d915dfca None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:22.276617 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 257/529] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:22.276617 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b5cfc-077a-46af-9f38-1361d915dfca None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:22.277267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 273/530] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:22.278449 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26683407-f6a0-4be5-8b35-03a2966cc566 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:22.380864 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b5cfc-077a-46af-9f38-1361d915dfca None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:22.409080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26683407-f6a0-4be5-8b35-03a2966cc566 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:22.413900 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:22.426032 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a08d2-dddd-4aa4-a77b-f84711a1dfd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:22.427631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a08d2-dddd-4aa4-a77b-f84711a1dfd3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:22.428145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a08d2-dddd-4aa4-a77b-f84711a1dfd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:22.428641 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a08d2-dddd-4aa4-a77b-f84711a1dfd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:22.449092 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71a08d2-dddd-4aa4-a77b-f84711a1dfd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:22.451808 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71a08d2-dddd-4aa4-a77b-f84711a1dfd3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:22.455609 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e89abd-22f9-495a-8a85-69a93d469268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:22.456419 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e89abd-22f9-495a-8a85-69a93d469268 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:22.456702 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e89abd-22f9-495a-8a85-69a93d469268 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:22.456977 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e89abd-22f9-495a-8a85-69a93d469268 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:22.464590 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e89abd-22f9-495a-8a85-69a93d469268 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:22.466198 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e89abd-22f9-495a-8a85-69a93d469268 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:22.475018 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-f71a08d2-dddd-4aa4-a77b-f84711a1dfd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:22.475018 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:22.475018 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:22.475018 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:22.475018 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:22.475018 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:22.475018 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:22.475018 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:22.475018 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:22.475018 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:22.475018 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:22.475018 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:22.475018 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:22.475018 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:22.475018 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:22.475942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:22.475942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:22.475942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:22.475942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:22.475942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:22.475942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:22.475942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:22.475942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:22.475942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:22.475942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:22.475942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:22.475942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:22.475942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:22.475942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:22.475942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:22.475942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:22.475942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:22.476833 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:22.476833 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:22.476833 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:22.476833 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:22.479236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 258/531] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:22.480325 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a08d2-dddd-4aa4-a77b-f84711a1dfd3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:22.482604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-a3e89abd-22f9-495a-8a85-69a93d469268 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:22.482604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:22.482604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:22.482604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:22.482604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:22.482604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:22.482604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:22.482604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:22.482604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:22.482604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:22.482604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:22.482604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:22.482604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:51:22.482604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:51:22.482604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:51:22.483461 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:51:22.483461 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:22.483461 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:22.483461 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:22.483461 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:22.483461 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:22.483461 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:22.483461 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:22.483461 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:22.483461 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:22.483461 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:22.483461 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:22.483461 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:22.483461 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:22.483461 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:22.483461 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:22.483461 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:22.484354 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:51:22.484354 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:51:22.484354 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:51:22.484354 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:51:22.484354 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:51:22.484354 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:51:22.484354 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:22.484354 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:22.484796 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 274/532] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:51:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:22.485486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e89abd-22f9-495a-8a85-69a93d469268 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:22.501496 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:22.616840 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a08d2-dddd-4aa4-a77b-f84711a1dfd3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:22.623231 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e89abd-22f9-495a-8a85-69a93d469268 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:22.667484 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf543d-dec7-4e06-a7ed-452a38b16d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:22.668669 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf543d-dec7-4e06-a7ed-452a38b16d83 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:22.668789 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964392c5-7f2c-4975-b064-e60fe2a97fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:22.669287 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf543d-dec7-4e06-a7ed-452a38b16d83 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:22.669437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964392c5-7f2c-4975-b064-e60fe2a97fee None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:22.669894 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964392c5-7f2c-4975-b064-e60fe2a97fee None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:22.670064 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf543d-dec7-4e06-a7ed-452a38b16d83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:22.670287 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964392c5-7f2c-4975-b064-e60fe2a97fee None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:22.671296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964392c5-7f2c-4975-b064-e60fe2a97fee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:22.671690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964392c5-7f2c-4975-b064-e60fe2a97fee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:22.673958 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964392c5-7f2c-4975-b064-e60fe2a97fee None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:22.679465 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf543d-dec7-4e06-a7ed-452a38b16d83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:22.681239 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf543d-dec7-4e06-a7ed-452a38b16d83 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:22.689038 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 275/533] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:22.690157 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964392c5-7f2c-4975-b064-e60fe2a97fee None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:22.704550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-88bf543d-dec7-4e06-a7ed-452a38b16d83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:22.704550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:22.704550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:22.704550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:22.704550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:22.704550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:22.704550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:22.704550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:22.704550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:22.704550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:22.704550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:22.704550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:22.704550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:51:22.704550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:51:22.704550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:51:22.705953 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:51:22.705953 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:22.705953 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:22.705953 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:22.705953 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:22.705953 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:22.705953 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:22.705953 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:22.705953 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:22.705953 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:22.705953 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:22.705953 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:22.705953 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:22.705953 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:22.705953 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:22.705953 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:22.705953 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:22.707326 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:51:22.707326 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:51:22.707326 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:51:22.707326 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:51:22.707326 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:51:22.707326 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:51:22.707326 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:22.707326 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:22.711185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 259/534] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:51:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:22.712541 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf543d-dec7-4e06-a7ed-452a38b16d83 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:22.719929 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:22.802038 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:22.821095 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964392c5-7f2c-4975-b064-e60fe2a97fee None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:22.837209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf543d-dec7-4e06-a7ed-452a38b16d83 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:22.871142 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2dc02-b663-4185-8775-c7469d2d032f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:22.872320 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2dc02-b663-4185-8775-c7469d2d032f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:22.872879 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2dc02-b663-4185-8775-c7469d2d032f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:22.873438 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2dc02-b663-4185-8775-c7469d2d032f None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:22.874545 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d2dc02-b663-4185-8775-c7469d2d032f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:22.875179 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d2dc02-b663-4185-8775-c7469d2d032f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:22.875726 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25925b0-323e-4ad0-a6d6-737f140ab6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:22.876578 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25925b0-323e-4ad0-a6d6-737f140ab6ca None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:22.876868 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25925b0-323e-4ad0-a6d6-737f140ab6ca None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:22.877146 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25925b0-323e-4ad0-a6d6-737f140ab6ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:22.878167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d2dc02-b663-4185-8775-c7469d2d032f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:22.884885 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25925b0-323e-4ad0-a6d6-737f140ab6ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:22.886394 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25925b0-323e-4ad0-a6d6-737f140ab6ca None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:22.894589 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 276/535] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:51:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:22.895898 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2dc02-b663-4185-8775-c7469d2d032f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:22.901406 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-d25925b0-323e-4ad0-a6d6-737f140ab6ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:22.901406 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:22.901406 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:22.901406 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:22.901406 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:22.901406 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:22.901406 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:22.901406 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:22.901406 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:22.901406 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:22.901406 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:22.901406 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:22.901406 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:51:22.901406 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:51:22.901406 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:51:22.902418 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:51:22.902418 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:22.902418 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:22.902418 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:22.902418 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:22.902418 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:22.902418 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:22.902418 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:22.902418 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:22.902418 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:22.902418 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:22.902418 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:22.902418 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:22.902418 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:22.902418 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:22.902418 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:22.902418 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:22.903857 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:51:22.903857 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:51:22.903857 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:51:22.903857 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:51:22.903857 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:51:22.903857 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:51:22.903857 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:22.903857 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:22.903857 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 260/536] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:51:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:22.903857 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25925b0-323e-4ad0-a6d6-737f140ab6ca None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:23.038097 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25925b0-323e-4ad0-a6d6-737f140ab6ca None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:23.038585 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:23.061597 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2dc02-b663-4185-8775-c7469d2d032f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:23.099370 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:23.119429 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd7c93-aa9c-4c30-a804-230338c38a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:23.120291 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd7c93-aa9c-4c30-a804-230338c38a9a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:23.120649 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd7c93-aa9c-4c30-a804-230338c38a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:23.120989 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd7c93-aa9c-4c30-a804-230338c38a9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:23.121880 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd7c93-aa9c-4c30-a804-230338c38a9a None admin] RBAC: Adding query filter params (role.id=64d94d0d5a7640818c21aa5a622624ab, scope.project.id=bb1ac00308714c39a79aae09fbb73021, user.id=87c0598af7714aeba08fffbb6805a49e) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:23.122224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd7c93-aa9c-4c30-a804-230338c38a9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:23.123937 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd7c93-aa9c-4c30-a804-230338c38a9a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:23.128377 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d110a2f-ddc7-452f-9124-471fbcc62b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:23.129616 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d110a2f-ddc7-452f-9124-471fbcc62b26 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:23.130148 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d110a2f-ddc7-452f-9124-471fbcc62b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:23.130719 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d110a2f-ddc7-452f-9124-471fbcc62b26 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:23.132231 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d110a2f-ddc7-452f-9124-471fbcc62b26 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:23.133005 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d110a2f-ddc7-452f-9124-471fbcc62b26 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:23.135863 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d110a2f-ddc7-452f-9124-471fbcc62b26 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:23.138014 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 261/537] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:51:22 2022] GET /identity/v3/role_assignments?user.id=87c0598af7714aeba08fffbb6805a49e&scope.project.id=bb1ac00308714c39a79aae09fbb73021&role.id=64d94d0d5a7640818c21aa5a622624ab => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:23.139012 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd7c93-aa9c-4c30-a804-230338c38a9a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:23.163962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 277/538] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:51:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:23.170821 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d110a2f-ddc7-452f-9124-471fbcc62b26 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:23.345069 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d110a2f-ddc7-452f-9124-471fbcc62b26 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:23.361172 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd7c93-aa9c-4c30-a804-230338c38a9a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:23.402759 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399a5b76-9f68-427d-839f-e56342097b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:23.403935 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a5b76-9f68-427d-839f-e56342097b68 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:23.404300 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a5b76-9f68-427d-839f-e56342097b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:23.404658 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a5b76-9f68-427d-839f-e56342097b68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:23.405616 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399a5b76-9f68-427d-839f-e56342097b68 None admin] RBAC: Adding query filter params (role.id=64d94d0d5a7640818c21aa5a622624ab, scope.project.id=bb1ac00308714c39a79aae09fbb73021, user.id=d73a7d5f37e144768d7cc53f13cd5157) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:23.406336 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399a5b76-9f68-427d-839f-e56342097b68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:23.408817 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399a5b76-9f68-427d-839f-e56342097b68 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:23.413736 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f577e7e-70c1-4ae6-9511-e02043a8353a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:23.414627 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f577e7e-70c1-4ae6-9511-e02043a8353a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:23.415053 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f577e7e-70c1-4ae6-9511-e02043a8353a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:23.416020 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f577e7e-70c1-4ae6-9511-e02043a8353a None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:23.416997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f577e7e-70c1-4ae6-9511-e02043a8353a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:23.417769 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f577e7e-70c1-4ae6-9511-e02043a8353a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:23.419391 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f577e7e-70c1-4ae6-9511-e02043a8353a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:23.424941 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 278/539] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:51:23 2022] GET /identity/v3/role_assignments?user.id=d73a7d5f37e144768d7cc53f13cd5157&scope.project.id=bb1ac00308714c39a79aae09fbb73021&role.id=64d94d0d5a7640818c21aa5a622624ab => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:23.424941 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e2b37-288e-47fd-b819-3c32408f5b14 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:23.424941 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e2b37-288e-47fd-b819-3c32408f5b14 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:23.424941 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e2b37-288e-47fd-b819-3c32408f5b14 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:23.425564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 279/540] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:23.435092 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b150b10-af43-4c68-8f65-4b3bff6f3aa0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:23.435395 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b150b10-af43-4c68-8f65-4b3bff6f3aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:23.435662 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b150b10-af43-4c68-8f65-4b3bff6f3aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:23.470638 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8b150b10-af43-4c68-8f65-4b3bff6f3aa0 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:23.472030 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8b150b10-af43-4c68-8f65-4b3bff6f3aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:23.477469 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 262/541] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:51:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:23.484655 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f577e7e-70c1-4ae6-9511-e02043a8353a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:23.494017 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:23.572208 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:23.630375 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f577e7e-70c1-4ae6-9511-e02043a8353a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:23.671618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df109ede-1449-403d-bd1b-f7359caf2d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:23.673073 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df109ede-1449-403d-bd1b-f7359caf2d4a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:23.673560 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df109ede-1449-403d-bd1b-f7359caf2d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:23.674025 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df109ede-1449-403d-bd1b-f7359caf2d4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:23.675656 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df109ede-1449-403d-bd1b-f7359caf2d4a None admin] RBAC: Adding query filter params (role.id=64d94d0d5a7640818c21aa5a622624ab, scope.project.id=bb1ac00308714c39a79aae09fbb73021, user.id=1654fdf5ba6b46ba8d9b86a83a76a4b5) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:23.676126 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df109ede-1449-403d-bd1b-f7359caf2d4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:23.677768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df109ede-1449-403d-bd1b-f7359caf2d4a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:23.685066 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 263/542] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:51:23 2022] GET /identity/v3/role_assignments?user.id=1654fdf5ba6b46ba8d9b86a83a76a4b5&scope.project.id=bb1ac00308714c39a79aae09fbb73021&role.id=64d94d0d5a7640818c21aa5a622624ab => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:23.701350 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 280/543] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:51:23.733146 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df109ede-1449-403d-bd1b-f7359caf2d4a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:23.911707 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df109ede-1449-403d-bd1b-f7359caf2d4a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:23.941458 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:23.960854 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b54d3-44d8-4b42-a5fc-6472be897fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:23.961690 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b54d3-44d8-4b42-a5fc-6472be897fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:23.961980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b54d3-44d8-4b42-a5fc-6472be897fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:23.962314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b54d3-44d8-4b42-a5fc-6472be897fe6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:23.973814 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756578f-4dc0-48cc-9c17-8f46c9a91db2 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:23.974114 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756578f-4dc0-48cc-9c17-8f46c9a91db2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:23.974378 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756578f-4dc0-48cc-9c17-8f46c9a91db2 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:23.975754 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 281/544] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:23.981012 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ed024-ae64-4f1b-8e5e-0d2d82f50897 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:23.981306 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ed024-ae64-4f1b-8e5e-0d2d82f50897 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:23.981603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ed024-ae64-4f1b-8e5e-0d2d82f50897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:23.992938 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b54d3-44d8-4b42-a5fc-6472be897fe6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:24.020349 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-531b54d3-44d8-4b42-a5fc-6472be897fe6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:24.020349 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:24.020349 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:24.020349 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:24.020349 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:24.020349 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:24.020349 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:24.020349 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:24.020349 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:24.020349 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:24.020349 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:24.020349 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:24.020349 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:24.020349 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:24.020349 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:24.020349 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:24.022000 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:24.022000 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:24.022000 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:24.022000 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:24.022000 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:24.022000 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:24.022000 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:24.022000 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:24.022000 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:24.022000 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:24.022000 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:24.022000 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:24.022000 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:24.022000 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:24.022000 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:24.022000 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:24.023454 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:24.023454 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:24.023454 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:24.023454 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:24.023454 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:24.023454 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:24.023454 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:24.023454 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:24.023454 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:24.023454 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:24.023454 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:24.023454 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:24.031279 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 264/545] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 15:51:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:24.033622 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ac8ed024-ae64-4f1b-8e5e-0d2d82f50897 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:24.035753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ac8ed024-ae64-4f1b-8e5e-0d2d82f50897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:24.040560 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b54d3-44d8-4b42-a5fc-6472be897fe6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:24.190610 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 282/546] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 15:51:24.232529 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b54d3-44d8-4b42-a5fc-6472be897fe6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:24.250207 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ed024-ae64-4f1b-8e5e-0d2d82f50897 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:24.302355 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232a71f-741d-49d6-b9f9-7dacb724168d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:24.303038 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232a71f-741d-49d6-b9f9-7dacb724168d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:24.303319 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232a71f-741d-49d6-b9f9-7dacb724168d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:24.303593 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232a71f-741d-49d6-b9f9-7dacb724168d None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:24.304362 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232a71f-741d-49d6-b9f9-7dacb724168d None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:24.304670 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232a71f-741d-49d6-b9f9-7dacb724168d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:24.306110 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232a71f-741d-49d6-b9f9-7dacb724168d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:24.326997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 265/547] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 15:51:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:24.332823 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232a71f-741d-49d6-b9f9-7dacb724168d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:24.359085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:24.383952 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:24.447842 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232a71f-741d-49d6-b9f9-7dacb724168d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:24.469558 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ed024-ae64-4f1b-8e5e-0d2d82f50897 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:24.483550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5371f75-99dd-4a65-852a-c9e537d90c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:24.484418 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5371f75-99dd-4a65-852a-c9e537d90c26 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:24.484700 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5371f75-99dd-4a65-852a-c9e537d90c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:24.485545 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5371f75-99dd-4a65-852a-c9e537d90c26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:24.497730 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5371f75-99dd-4a65-852a-c9e537d90c26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:24.499210 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5371f75-99dd-4a65-852a-c9e537d90c26 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:24.517881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-c5371f75-99dd-4a65-852a-c9e537d90c26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:24.517881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:24.517881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:24.517881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:24.517881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:24.517881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:24.517881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:24.517881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:24.517881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:24.517881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:24.517881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:24.517881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:24.517881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:24.517881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:24.517881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:24.518845 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:24.518845 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:24.518845 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:24.518845 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:24.518845 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:24.518845 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:24.518845 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:24.518845 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:24.518845 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:24.518845 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:24.518845 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:24.518845 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:24.518845 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:24.518845 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:24.518845 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:24.518845 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:24.518845 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:24.519705 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:24.519705 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:24.519705 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:24.519705 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:24.519705 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 266/548] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:24.525733 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5371f75-99dd-4a65-852a-c9e537d90c26 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:24.563467 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe32bb8-f8e3-42aa-baf5-bb30a7a3d900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:24.564281 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe32bb8-f8e3-42aa-baf5-bb30a7a3d900 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:24.564560 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe32bb8-f8e3-42aa-baf5-bb30a7a3d900 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:24.564833 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe32bb8-f8e3-42aa-baf5-bb30a7a3d900 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:24.583399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe32bb8-f8e3-42aa-baf5-bb30a7a3d900 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:24.591215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-6fe32bb8-f8e3-42aa-baf5-bb30a7a3d900 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:24.591215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:24.591215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:24.591215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:24.591215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:24.591215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:24.591215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:24.591215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:24.591215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:24.591215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:24.591215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:24.591215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:24.591215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:24.591215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:24.591215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:24.591215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:24.592150 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:24.592150 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:24.592150 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:24.592150 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:24.592150 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:24.592150 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:24.592150 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:24.592150 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:24.592150 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:24.592150 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:24.592150 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:24.592150 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:24.592150 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:24.592150 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:24.592150 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:24.592150 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:24.593557 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:24.593557 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:24.593557 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:24.593557 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:24.593557 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:24.593557 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:24.593557 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:24.593557 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:24.593557 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:24.593557 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:24.593557 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:24.593557 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:24.599398 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 283/549] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 15:51:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:51:24.609795 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe32bb8-f8e3-42aa-baf5-bb30a7a3d900 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:24.694869 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5371f75-99dd-4a65-852a-c9e537d90c26 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:24.697928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:24.749216 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe32bb8-f8e3-42aa-baf5-bb30a7a3d900 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:24.755208 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c7123-a27b-4331-a547-7595a616ea5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:24.755685 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c7123-a27b-4331-a547-7595a616ea5b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:24.755975 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c7123-a27b-4331-a547-7595a616ea5b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:24.756258 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c7123-a27b-4331-a547-7595a616ea5b None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:24.757000 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4c7123-a27b-4331-a547-7595a616ea5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:24.757314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4c7123-a27b-4331-a547-7595a616ea5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:24.758772 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4c7123-a27b-4331-a547-7595a616ea5b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:24.776049 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 267/550] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:24.778795 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:24.784429 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c7123-a27b-4331-a547-7595a616ea5b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:24.795922 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6047f7-ed70-4144-831b-de6da490ccc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:24.799372 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6047f7-ed70-4144-831b-de6da490ccc4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:24.803465 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6047f7-ed70-4144-831b-de6da490ccc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:24.804111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6047f7-ed70-4144-831b-de6da490ccc4 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:24.805237 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6047f7-ed70-4144-831b-de6da490ccc4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:24.805799 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6047f7-ed70-4144-831b-de6da490ccc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:24.807423 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6047f7-ed70-4144-831b-de6da490ccc4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:24.828242 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 284/551] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 15:51:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:51:24.833809 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6047f7-ed70-4144-831b-de6da490ccc4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:24.996999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6047f7-ed70-4144-831b-de6da490ccc4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:25.038237 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2cd22d-84de-4e7e-b3e3-8429797cae56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:25.040632 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2cd22d-84de-4e7e-b3e3-8429797cae56 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:25.040632 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2cd22d-84de-4e7e-b3e3-8429797cae56 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:25.040632 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2cd22d-84de-4e7e-b3e3-8429797cae56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:25.054516 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c7123-a27b-4331-a547-7595a616ea5b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:25.056171 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2cd22d-84de-4e7e-b3e3-8429797cae56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:25.059259 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2cd22d-84de-4e7e-b3e3-8429797cae56 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:25.082541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-ff2cd22d-84de-4e7e-b3e3-8429797cae56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:25.082541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:25.082541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:25.082541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:25.082541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:25.082541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:25.082541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:25.082541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:25.082541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:25.082541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:25.082541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:25.082541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:25.082541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:25.082541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:25.082541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:25.084328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:25.084328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:25.084328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:25.084328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:25.084328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:25.084328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:25.084328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:25.084328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:25.084328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:25.084328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:25.084328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:25.084328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:25.084328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:25.084328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:25.084328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:25.084328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:25.084328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:25.086201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:25.086201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:25.086201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:25.086201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:25.086201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 285/552] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:25.089728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2cd22d-84de-4e7e-b3e3-8429797cae56 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:25.144829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:25.163192 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdbbd6b-7a14-43b6-be21-d239fb5b39e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:25.164347 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbbd6b-7a14-43b6-be21-d239fb5b39e6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:25.164924 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbbd6b-7a14-43b6-be21-d239fb5b39e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:25.165456 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbbd6b-7a14-43b6-be21-d239fb5b39e6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:25.189261 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdbbd6b-7a14-43b6-be21-d239fb5b39e6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:25.191377 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdbbd6b-7a14-43b6-be21-d239fb5b39e6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:25.217227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ccdbbd6b-7a14-43b6-be21-d239fb5b39e6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 15:51:25.217227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:25.217227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:25.217227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:25.217227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:25.217227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:25.217227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:25.217227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:25.217227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:25.217227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:25.217227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:25.217227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:25.217227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:51:25.217227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:51:25.217227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:51:25.217227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:51:25.219358 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:25.219358 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:25.219358 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:51:25.219358 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:25.219358 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:51:25.219358 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:25.219358 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:25.219358 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:25.219358 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:25.219358 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:25.219358 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:25.219358 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:25.219358 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:25.219358 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:25.219358 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:25.219358 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:25.219358 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:25.219358 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:25.221488 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:25.221488 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:25.221488 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:51:25.221488 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:51:25.221488 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:51:25.221488 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:51:25.221488 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:51:25.221488 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:51:25.221488 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 15:51:25.221488 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:25.226532 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 268/553] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:51:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 443 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:51:25.235641 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdbbd6b-7a14-43b6-be21-d239fb5b39e6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:25.262535 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2cd22d-84de-4e7e-b3e3-8429797cae56 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:25.282480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:25.303360 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d9368-50f8-4526-a603-0a4b04db319e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:25.304460 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d9368-50f8-4526-a603-0a4b04db319e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:25.304892 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d9368-50f8-4526-a603-0a4b04db319e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:25.305310 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d9368-50f8-4526-a603-0a4b04db319e None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:25.306248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371d9368-50f8-4526-a603-0a4b04db319e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:25.306699 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371d9368-50f8-4526-a603-0a4b04db319e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:25.308258 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371d9368-50f8-4526-a603-0a4b04db319e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:25.324704 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 286/554] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:25.330108 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d9368-50f8-4526-a603-0a4b04db319e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:25.477961 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d9368-50f8-4526-a603-0a4b04db319e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:25.485946 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:25.506791 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdbbd6b-7a14-43b6-be21-d239fb5b39e6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:25.547078 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f7d8d-26c6-46c8-882f-f3918fd6e623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:25.547811 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f7d8d-26c6-46c8-882f-f3918fd6e623 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:25.548096 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f7d8d-26c6-46c8-882f-f3918fd6e623 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:25.548375 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f7d8d-26c6-46c8-882f-f3918fd6e623 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:25.549284 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1f7d8d-26c6-46c8-882f-f3918fd6e623 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:25.549620 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1f7d8d-26c6-46c8-882f-f3918fd6e623 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:25.551896 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1f7d8d-26c6-46c8-882f-f3918fd6e623 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:25.566737 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8d966-e757-4b5e-abca-09ce6a42c5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:25.567912 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8d966-e757-4b5e-abca-09ce6a42c5cd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:25.568394 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8d966-e757-4b5e-abca-09ce6a42c5cd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:25.568861 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8d966-e757-4b5e-abca-09ce6a42c5cd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:25.575968 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 269/555] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Jul 21 15:51:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:25.577751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b8d966-e757-4b5e-abca-09ce6a42c5cd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:25.577865 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4746a-fdca-4083-b0b7-154137dadb16 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:25.578032 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4746a-fdca-4083-b0b7-154137dadb16 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:25.578313 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4746a-fdca-4083-b0b7-154137dadb16 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:25.583807 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b8d966-e757-4b5e-abca-09ce6a42c5cd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:25.583969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 270/556] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:25.583969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf4746a-fdca-4083-b0b7-154137dadb16 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:25.610436 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-04b8d966-e757-4b5e-abca-09ce6a42c5cd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 15:51:25.610436 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:25.610436 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:25.610436 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:25.610436 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:25.610436 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:25.610436 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:25.610436 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:25.610436 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:25.610436 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:25.610436 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:25.610436 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:25.610436 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:51:25.610436 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:51:25.610436 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:51:25.610436 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:51:25.611911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:25.611911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:25.611911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:51:25.611911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:25.611911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:51:25.611911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:25.611911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:25.611911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:25.611911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:25.611911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:25.611911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:25.611911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:25.611911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:25.611911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:25.611911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:25.611911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:25.611911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:25.611911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:25.613362 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:25.613362 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:25.613362 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:51:25.613362 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:51:25.613362 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:51:25.613362 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:51:25.613362 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:51:25.613362 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:51:25.613362 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 15:51:25.613362 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:25.613362 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 287/557] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jul 21 15:51:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:25.614031 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db21be90-f9ce-46c8-a335-108380518608 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:25.614031 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db21be90-f9ce-46c8-a335-108380518608 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:25.614542 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db21be90-f9ce-46c8-a335-108380518608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:25.641636 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-db21be90-f9ce-46c8-a335-108380518608 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:25.643538 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-db21be90-f9ce-46c8-a335-108380518608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:25.706039 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:25.726573 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf4746a-fdca-4083-b0b7-154137dadb16 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:25.736050 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 288/558] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:51:25.737078 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db21be90-f9ce-46c8-a335-108380518608 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:25.791796 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff020230-b8b5-4fb9-8fc0-fac32ad6e646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:25.792467 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff020230-b8b5-4fb9-8fc0-fac32ad6e646 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:25.792756 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff020230-b8b5-4fb9-8fc0-fac32ad6e646 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:25.793040 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff020230-b8b5-4fb9-8fc0-fac32ad6e646 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:25.865366 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:25.875024 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db21be90-f9ce-46c8-a335-108380518608 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:25.895800 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff020230-b8b5-4fb9-8fc0-fac32ad6e646 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:25.897555 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff020230-b8b5-4fb9-8fc0-fac32ad6e646 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:25.916485 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373457b-ce0e-4b5b-be34-ca10a9bf387c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:25.917175 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373457b-ce0e-4b5b-be34-ca10a9bf387c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:25.917463 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373457b-ce0e-4b5b-be34-ca10a9bf387c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:25.917784 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373457b-ce0e-4b5b-be34-ca10a9bf387c None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:25.918708 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373457b-ce0e-4b5b-be34-ca10a9bf387c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:25.919032 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373457b-ce0e-4b5b-be34-ca10a9bf387c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:25.920499 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373457b-ce0e-4b5b-be34-ca10a9bf387c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:25.940560 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 289/559] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jul 21 15:51:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:25.946046 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373457b-ce0e-4b5b-be34-ca10a9bf387c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:25.982403 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:26.019904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 271/560] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:51:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 15:51:26.021021 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff020230-b8b5-4fb9-8fc0-fac32ad6e646 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:26.088343 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373457b-ce0e-4b5b-be34-ca10a9bf387c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:26.128848 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf856f-7b14-473a-980e-fa09741ff385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:26.129921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf856f-7b14-473a-980e-fa09741ff385 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:26.130373 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf856f-7b14-473a-980e-fa09741ff385 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:26.130651 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf856f-7b14-473a-980e-fa09741ff385 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:26.134873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff020230-b8b5-4fb9-8fc0-fac32ad6e646 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:26.142472 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf856f-7b14-473a-980e-fa09741ff385 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:26.150329 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-a3cf856f-7b14-473a-980e-fa09741ff385 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:26.150329 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:26.150329 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:26.150329 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:26.150329 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:26.150329 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:26.150329 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:26.150329 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:26.150329 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:26.150329 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:26.150329 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:26.150329 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:26.150329 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:26.150329 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:26.150329 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:26.150329 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:26.151826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:26.151826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:26.151826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:26.151826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:26.151826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:26.151826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:26.151826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:26.151826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:26.151826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:26.151826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:26.151826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:26.151826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:26.151826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:26.151826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:26.151826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:26.151826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:26.153270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:26.153270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:26.153270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:26.153270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:26.153270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:26.153270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:26.153270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:26.153270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:26.153270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:26.153270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:26.153270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:26.153270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:26.153270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 290/561] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 15:51:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:26.153270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf856f-7b14-473a-980e-fa09741ff385 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:26.168852 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:26.177837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb9cd9-224f-4ec0-9563-d1487a7cfffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:26.178635 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb9cd9-224f-4ec0-9563-d1487a7cfffd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:26.178917 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb9cd9-224f-4ec0-9563-d1487a7cfffd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:26.179192 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb9cd9-224f-4ec0-9563-d1487a7cfffd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:26.190335 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb9cd9-224f-4ec0-9563-d1487a7cfffd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:26.191805 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb9cd9-224f-4ec0-9563-d1487a7cfffd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:26.210203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-18fb9cd9-224f-4ec0-9563-d1487a7cfffd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:26.210203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:26.210203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:26.210203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:26.210203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:26.210203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:26.210203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:26.210203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:26.210203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:26.210203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:26.210203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:26.210203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:26.210203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:26.210203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:26.210203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:26.211314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:26.211314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:26.211314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:26.211314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:26.211314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:26.211314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:26.211314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:26.211314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:26.211314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:26.211314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:26.211314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:26.211314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:26.211314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:26.211314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:26.211314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:26.211314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:26.211314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:26.213024 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:26.213024 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:26.213024 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:26.213024 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:26.213024 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 272/562] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:26.213024 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b70522-d40a-40fa-8b1f-6f5e422da290 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:26.213767 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b70522-d40a-40fa-8b1f-6f5e422da290 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:26.213767 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b70522-d40a-40fa-8b1f-6f5e422da290 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:26.214910 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 273/563] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:26.215619 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b70522-d40a-40fa-8b1f-6f5e422da290 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:26.244180 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:26.266674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf856f-7b14-473a-980e-fa09741ff385 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:26.308129 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c8e17d-a9aa-4184-a218-822256c61820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:26.308851 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c8e17d-a9aa-4184-a218-822256c61820 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:26.309132 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c8e17d-a9aa-4184-a218-822256c61820 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:26.309403 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c8e17d-a9aa-4184-a218-822256c61820 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:26.321998 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c8e17d-a9aa-4184-a218-822256c61820 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:26.323482 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c8e17d-a9aa-4184-a218-822256c61820 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:26.331260 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b70522-d40a-40fa-8b1f-6f5e422da290 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:26.342016 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-f4c8e17d-a9aa-4184-a218-822256c61820 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:26.342016 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:26.342016 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:26.342016 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:26.342016 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:26.342016 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:26.342016 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:26.342016 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:26.342016 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:26.342016 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:26.342016 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:26.342016 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:26.342016 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:26.342016 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:26.342016 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:26.343812 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:26.343812 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:26.343812 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:26.343812 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:26.343812 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:26.343812 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:26.343812 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:26.343812 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:26.343812 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:26.343812 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:26.343812 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:26.343812 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:26.343812 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:26.343812 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:26.343812 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:26.343812 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:26.343812 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:26.345887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:26.345887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:26.345887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:26.345887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:26.345887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 291/564] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:26.345887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a04b0f-193e-40b2-9ca3-98089d254a18 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:26.345887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a04b0f-193e-40b2-9ca3-98089d254a18 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:26.345887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a04b0f-193e-40b2-9ca3-98089d254a18 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:26.346881 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 292/565] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:26.347999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba0084-a637-45dd-a9dc-2e9f049a5938 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:26.348650 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba0084-a637-45dd-a9dc-2e9f049a5938 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:26.348650 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba0084-a637-45dd-a9dc-2e9f049a5938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:26.366887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-70ba0084-a637-45dd-a9dc-2e9f049a5938 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:26.368101 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-70ba0084-a637-45dd-a9dc-2e9f049a5938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:26.374217 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a6b09-56a7-4e44-a03c-4d6f3ccdf0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:26.374889 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a6b09-56a7-4e44-a03c-4d6f3ccdf0fe None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:26.375173 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a6b09-56a7-4e44-a03c-4d6f3ccdf0fe None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:26.376012 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a6b09-56a7-4e44-a03c-4d6f3ccdf0fe None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:26.376771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a6b09-56a7-4e44-a03c-4d6f3ccdf0fe None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:26.377075 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a6b09-56a7-4e44-a03c-4d6f3ccdf0fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:26.378493 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a6b09-56a7-4e44-a03c-4d6f3ccdf0fe None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:26.387289 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 274/566] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 15:51:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:26.388030 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a6b09-56a7-4e44-a03c-4d6f3ccdf0fe None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:26.431004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:26.446036 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 293/567] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:51:26.446898 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba0084-a637-45dd-a9dc-2e9f049a5938 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:26.498469 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:26.515963 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a6b09-56a7-4e44-a03c-4d6f3ccdf0fe None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:26.554508 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba0084-a637-45dd-a9dc-2e9f049a5938 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:26.559884 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b0bee-3b38-43fc-8c94-fdc89185333c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:26.560560 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b0bee-3b38-43fc-8c94-fdc89185333c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:26.560845 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b0bee-3b38-43fc-8c94-fdc89185333c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:26.561123 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b0bee-3b38-43fc-8c94-fdc89185333c None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:26.561904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b0bee-3b38-43fc-8c94-fdc89185333c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:26.562223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b0bee-3b38-43fc-8c94-fdc89185333c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:26.564232 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b0bee-3b38-43fc-8c94-fdc89185333c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:26.576951 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 275/568] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:26.578160 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98494b04-3cc9-45ea-8fe8-c724e3726684 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:26.578491 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98494b04-3cc9-45ea-8fe8-c724e3726684 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:26.578743 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98494b04-3cc9-45ea-8fe8-c724e3726684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:26.591637 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122055ac-892f-43c5-9426-b6311bbaaf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:26.592305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122055ac-892f-43c5-9426-b6311bbaaf05 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:26.592589 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122055ac-892f-43c5-9426-b6311bbaaf05 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:26.592864 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122055ac-892f-43c5-9426-b6311bbaaf05 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:26.593673 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122055ac-892f-43c5-9426-b6311bbaaf05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:26.593931 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122055ac-892f-43c5-9426-b6311bbaaf05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:26.594772 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-98494b04-3cc9-45ea-8fe8-c724e3726684 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:26.595352 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122055ac-892f-43c5-9426-b6311bbaaf05 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:26.595982 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-98494b04-3cc9-45ea-8fe8-c724e3726684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:26.607259 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 294/569] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:26.608108 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122055ac-892f-43c5-9426-b6311bbaaf05 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:26.666562 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:26.671904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 276/570] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:51:26.673663 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98494b04-3cc9-45ea-8fe8-c724e3726684 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:26.717539 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122055ac-892f-43c5-9426-b6311bbaaf05 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:26.755969 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad3db6-c1d5-4c39-90e8-720ad72b69c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:26.756776 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad3db6-c1d5-4c39-90e8-720ad72b69c4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:26.757064 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad3db6-c1d5-4c39-90e8-720ad72b69c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:26.757347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad3db6-c1d5-4c39-90e8-720ad72b69c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:26.767033 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:26.773534 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad3db6-c1d5-4c39-90e8-720ad72b69c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:26.775025 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad3db6-c1d5-4c39-90e8-720ad72b69c4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:26.791325 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98494b04-3cc9-45ea-8fe8-c724e3726684 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:26.792782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-20ad3db6-c1d5-4c39-90e8-720ad72b69c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:26.792782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:26.792782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:26.792782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:26.792782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:26.792782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:26.792782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:26.792782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:26.792782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:26.792782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:26.792782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:26.792782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:26.792782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:26.792782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:26.792782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:26.793734 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:26.793734 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:26.793734 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:26.793734 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:26.793734 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:26.793734 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:26.793734 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:26.793734 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:26.793734 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:26.793734 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:26.793734 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:26.793734 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:26.793734 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:26.793734 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:26.793734 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:26.793734 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:26.793734 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:26.794658 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:26.794658 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:26.794658 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:26.794658 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:26.794658 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 295/571] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:26.794955 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad3db6-c1d5-4c39-90e8-720ad72b69c4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:26.830480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfcd244-9d46-4934-b95b-25b0047b9c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:26.831380 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfcd244-9d46-4934-b95b-25b0047b9c98 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:26.831664 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfcd244-9d46-4934-b95b-25b0047b9c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:26.831939 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfcd244-9d46-4934-b95b-25b0047b9c98 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:26.847189 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfcd244-9d46-4934-b95b-25b0047b9c98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:26.855275 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-3dfcd244-9d46-4934-b95b-25b0047b9c98 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:26.855275 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:26.855275 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:26.855275 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:26.855275 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:26.855275 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:26.855275 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:26.855275 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:26.855275 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:26.855275 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:26.855275 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:26.855275 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:26.855275 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:26.855275 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:26.855275 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:26.855275 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:26.856468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:26.856468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:26.856468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:26.856468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:26.856468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:26.856468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:26.856468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:26.856468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:26.856468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:26.856468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:26.856468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:26.856468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:26.856468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:26.856468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:26.856468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:26.856468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:26.857387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:26.857387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:26.857387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:26.857387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:26.857387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:26.857387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:26.857387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:26.857387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:26.857387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:26.857387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:26.857387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:26.857387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:26.863005 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 277/572] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 15:51:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:26.863005 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfcd244-9d46-4934-b95b-25b0047b9c98 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:26.908055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad3db6-c1d5-4c39-90e8-720ad72b69c4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:26.910492 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:26.943612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced07839-7897-4863-b71f-394bd4ba05aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:26.944313 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced07839-7897-4863-b71f-394bd4ba05aa None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:26.944599 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced07839-7897-4863-b71f-394bd4ba05aa None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:26.944866 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced07839-7897-4863-b71f-394bd4ba05aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:26.952378 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced07839-7897-4863-b71f-394bd4ba05aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:26.953961 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced07839-7897-4863-b71f-394bd4ba05aa None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:26.968307 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-ced07839-7897-4863-b71f-394bd4ba05aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:26.968307 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:26.968307 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:26.968307 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:26.968307 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:26.968307 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:26.968307 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:26.968307 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:26.968307 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:26.968307 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:26.968307 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:26.968307 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:26.968307 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:51:26.968307 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:51:26.968307 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:51:26.970015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:51:26.970015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:26.970015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:26.970015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:26.970015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:26.970015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:26.970015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:26.970015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:26.970015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:26.970015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:26.970015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:26.970015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:26.970015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:26.970015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:26.970015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:26.970015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:26.970015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:26.970958 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:51:26.970958 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:51:26.970958 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:51:26.970958 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:51:26.970958 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:51:26.970958 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:51:26.970958 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:26.970958 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:26.970958 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 296/573] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:51:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:26.970958 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced07839-7897-4863-b71f-394bd4ba05aa None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:26.978443 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfcd244-9d46-4934-b95b-25b0047b9c98 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:27.013882 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3909cf-1374-47a2-b94c-bf750269fcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:27.014685 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3909cf-1374-47a2-b94c-bf750269fcda None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:27.014978 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3909cf-1374-47a2-b94c-bf750269fcda None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:27.015260 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3909cf-1374-47a2-b94c-bf750269fcda None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:27.018690 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:27.023508 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3909cf-1374-47a2-b94c-bf750269fcda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:27.025208 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3909cf-1374-47a2-b94c-bf750269fcda None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:27.040750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-fc3909cf-1374-47a2-b94c-bf750269fcda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:27.040750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:27.040750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:27.040750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:27.040750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:27.040750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:27.040750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:27.040750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:27.040750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:27.040750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:27.040750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:27.040750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:27.040750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:51:27.040750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:51:27.040750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:51:27.041787 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:51:27.041787 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:27.041787 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:27.041787 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:27.041787 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:27.041787 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:27.041787 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:27.041787 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:27.041787 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:27.041787 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:27.041787 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:27.041787 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:27.041787 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:27.041787 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:27.041787 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:27.041787 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:27.041787 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:27.042704 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:51:27.042704 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:51:27.042704 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:51:27.042704 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:51:27.042704 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:51:27.042704 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:51:27.042704 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:27.042704 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:27.042704 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 278/574] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:51:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:27.043206 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3909cf-1374-47a2-b94c-bf750269fcda None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:27.088848 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced07839-7897-4863-b71f-394bd4ba05aa None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:27.127030 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c67b4-442e-4ea1-8928-2e640a168971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:27.127955 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c67b4-442e-4ea1-8928-2e640a168971 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:27.128245 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c67b4-442e-4ea1-8928-2e640a168971 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:27.128527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c67b4-442e-4ea1-8928-2e640a168971 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:27.138928 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c67b4-442e-4ea1-8928-2e640a168971 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:27.145504 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3909cf-1374-47a2-b94c-bf750269fcda None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:27.147437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-401c67b4-442e-4ea1-8928-2e640a168971 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:27.147437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:27.147437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:27.147437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:27.147437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:27.147437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:27.147437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:27.147437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:27.147437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:27.147437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:27.147437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:27.147437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:27.147437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:27.147437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:27.147437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:27.147437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:27.148810 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:27.148810 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:27.148810 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:27.148810 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:27.148810 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:27.148810 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:27.148810 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:27.148810 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:27.148810 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:27.148810 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:27.148810 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:27.148810 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:27.148810 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:27.148810 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:27.148810 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:27.148810 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:27.149729 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:27.149729 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:27.149729 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:27.149729 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:27.149729 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:27.149729 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:27.149729 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:27.149729 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:27.149729 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:27.149729 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:27.149729 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:27.149729 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:27.149729 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 297/575] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 15:51:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:27.150448 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c67b4-442e-4ea1-8928-2e640a168971 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:27.178081 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:27.183223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148161be-452c-4805-86ce-6cebea12b60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:27.183897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148161be-452c-4805-86ce-6cebea12b60d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:27.184185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148161be-452c-4805-86ce-6cebea12b60d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:27.184785 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148161be-452c-4805-86ce-6cebea12b60d None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:27.185251 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148161be-452c-4805-86ce-6cebea12b60d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:27.185569 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148161be-452c-4805-86ce-6cebea12b60d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:27.186994 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148161be-452c-4805-86ce-6cebea12b60d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:27.199739 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 279/576] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:27.200305 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148161be-452c-4805-86ce-6cebea12b60d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:27.256767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c67b4-442e-4ea1-8928-2e640a168971 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:27.262966 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:27.301171 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004ed92-c016-4c89-96a5-a0259d84f875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:27.301883 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004ed92-c016-4c89-96a5-a0259d84f875 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:27.302163 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004ed92-c016-4c89-96a5-a0259d84f875 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:27.302441 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004ed92-c016-4c89-96a5-a0259d84f875 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:27.303193 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d004ed92-c016-4c89-96a5-a0259d84f875 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:27.303497 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d004ed92-c016-4c89-96a5-a0259d84f875 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:27.304913 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d004ed92-c016-4c89-96a5-a0259d84f875 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:27.315799 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 298/577] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 15:51:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:27.316408 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148161be-452c-4805-86ce-6cebea12b60d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:27.317083 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004ed92-c016-4c89-96a5-a0259d84f875 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:27.354937 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188f718-25b4-470c-9ec1-815ebc53c089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:27.355977 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188f718-25b4-470c-9ec1-815ebc53c089 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:27.356432 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188f718-25b4-470c-9ec1-815ebc53c089 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:27.356902 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188f718-25b4-470c-9ec1-815ebc53c089 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:27.358413 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1188f718-25b4-470c-9ec1-815ebc53c089 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:27.358972 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1188f718-25b4-470c-9ec1-815ebc53c089 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:27.362300 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1188f718-25b4-470c-9ec1-815ebc53c089 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:27.376316 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 280/578] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:51:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:27.376316 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188f718-25b4-470c-9ec1-815ebc53c089 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:27.407075 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:27.422748 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004ed92-c016-4c89-96a5-a0259d84f875 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:27.458317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2da9ee3-9e1a-46d2-9a9d-3677bff5dab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:27.458998 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da9ee3-9e1a-46d2-9a9d-3677bff5dab4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:27.459261 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da9ee3-9e1a-46d2-9a9d-3677bff5dab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:27.459535 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da9ee3-9e1a-46d2-9a9d-3677bff5dab4 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:27.460326 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2da9ee3-9e1a-46d2-9a9d-3677bff5dab4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:27.460634 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2da9ee3-9e1a-46d2-9a9d-3677bff5dab4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:27.462134 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2da9ee3-9e1a-46d2-9a9d-3677bff5dab4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:27.474974 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 299/579] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:51:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:27.475815 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2da9ee3-9e1a-46d2-9a9d-3677bff5dab4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:27.505212 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188f718-25b4-470c-9ec1-815ebc53c089 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:27.521688 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:27.541144 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235264b-5e61-4d01-b074-2f9e7364138a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:27.541836 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235264b-5e61-4d01-b074-2f9e7364138a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:27.542116 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235264b-5e61-4d01-b074-2f9e7364138a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:27.542380 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235264b-5e61-4d01-b074-2f9e7364138a None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:27.543128 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7235264b-5e61-4d01-b074-2f9e7364138a None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:27.543433 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7235264b-5e61-4d01-b074-2f9e7364138a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:27.544796 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7235264b-5e61-4d01-b074-2f9e7364138a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:27.554491 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 281/580] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 15:51:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:27.555375 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235264b-5e61-4d01-b074-2f9e7364138a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:27.587631 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2da9ee3-9e1a-46d2-9a9d-3677bff5dab4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:27.625505 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1250e8-88d2-4ec9-aa87-1a7689bb61aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:27.626393 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1250e8-88d2-4ec9-aa87-1a7689bb61aa None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:27.626676 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1250e8-88d2-4ec9-aa87-1a7689bb61aa None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:27.627012 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1250e8-88d2-4ec9-aa87-1a7689bb61aa None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:27.634823 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1250e8-88d2-4ec9-aa87-1a7689bb61aa None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:27.636323 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1250e8-88d2-4ec9-aa87-1a7689bb61aa None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:27.651500 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-eb1250e8-88d2-4ec9-aa87-1a7689bb61aa None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 15:51:27.651500 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:27.651500 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:27.651500 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:27.651500 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:27.651500 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:27.651500 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:27.651500 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:27.651500 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:27.651500 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:27.651500 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:27.651500 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:27.651500 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:51:27.651500 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:51:27.651500 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:51:27.651500 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:51:27.652769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:27.652769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:27.652769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:51:27.652769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:27.652769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:51:27.652769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:27.652769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:27.652769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:27.652769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:27.652769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:27.652769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:27.652769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:27.652769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:27.652769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:27.652769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:27.652769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:27.652769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:27.652769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:27.654151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:27.654151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:27.654151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:51:27.654151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:51:27.654151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:51:27.654151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:51:27.654151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:51:27.654151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:51:27.654151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 15:51:27.654151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:27.654151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 300/581] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:51:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:27.654151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1250e8-88d2-4ec9-aa87-1a7689bb61aa None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:27.664358 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:27.677390 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235264b-5e61-4d01-b074-2f9e7364138a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:27.713112 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecddfab-5103-4e44-b64e-39cef4027c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:27.713864 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecddfab-5103-4e44-b64e-39cef4027c57 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:27.714147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecddfab-5103-4e44-b64e-39cef4027c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:27.714427 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecddfab-5103-4e44-b64e-39cef4027c57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:27.725906 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecddfab-5103-4e44-b64e-39cef4027c57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:27.727367 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecddfab-5103-4e44-b64e-39cef4027c57 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:27.745469 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-fecddfab-5103-4e44-b64e-39cef4027c57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:27.745469 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:27.745469 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:27.745469 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:27.745469 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:27.745469 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:27.745469 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:27.745469 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:27.745469 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:27.745469 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:27.745469 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:27.745469 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:27.745469 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:27.745469 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:27.745469 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:27.746572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:27.746572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:27.746572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:27.746572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:27.746572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:27.746572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:27.746572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:27.746572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:27.746572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:27.746572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:27.746572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:27.746572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:27.746572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:27.746572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:27.746572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:27.746572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:27.746572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:27.752145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:27.752145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:27.752145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:27.752145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:27.752145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 282/582] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:27.752145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecddfab-5103-4e44-b64e-39cef4027c57 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:27.760074 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1250e8-88d2-4ec9-aa87-1a7689bb61aa None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:27.778505 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:27.794126 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294156af-0e7f-4614-b15b-7b8a08412ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:27.795045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294156af-0e7f-4614-b15b-7b8a08412ae2 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:27.795329 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294156af-0e7f-4614-b15b-7b8a08412ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:27.795602 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294156af-0e7f-4614-b15b-7b8a08412ae2 None admin] PATH_INFO: `/v3/projects/bb1ac00308714c39a79aae09fbb73021/users/092553a86f8848dba8c7271f5c0b2c46/roles/64d94d0d5a7640818c21aa5a622624ab` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:27.810812 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294156af-0e7f-4614-b15b-7b8a08412ae2 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb1ac00308714c39a79aae09fbb73021, user_id=092553a86f8848dba8c7271f5c0b2c46, role_id=64d94d0d5a7640818c21aa5a622624ab)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:27.812288 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294156af-0e7f-4614-b15b-7b8a08412ae2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:27.824908 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 301/583] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 15:51:27 2022] PUT /identity/v3/projects/bb1ac00308714c39a79aae09fbb73021/users/092553a86f8848dba8c7271f5c0b2c46/roles/64d94d0d5a7640818c21aa5a622624ab => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 15:51:27.825763 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294156af-0e7f-4614-b15b-7b8a08412ae2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:27.853509 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecddfab-5103-4e44-b64e-39cef4027c57 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:27.893959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e26682-f9e5-4d18-aea8-8b6a7d597075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:27.894184 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:27.894679 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e26682-f9e5-4d18-aea8-8b6a7d597075 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:27.895040 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e26682-f9e5-4d18-aea8-8b6a7d597075 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:27.895244 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e26682-f9e5-4d18-aea8-8b6a7d597075 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:27.896178 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e26682-f9e5-4d18-aea8-8b6a7d597075 None admin] RBAC: Adding query filter params (role.id=64d94d0d5a7640818c21aa5a622624ab, scope.project.id=bb1ac00308714c39a79aae09fbb73021, user.id=f5bfae2cb30843b1b1a0343efef38638) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:27.896503 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e26682-f9e5-4d18-aea8-8b6a7d597075 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:27.898123 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e26682-f9e5-4d18-aea8-8b6a7d597075 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:27.904700 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 283/584] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:51:27 2022] GET /identity/v3/role_assignments?user.id=f5bfae2cb30843b1b1a0343efef38638&scope.project.id=bb1ac00308714c39a79aae09fbb73021&role.id=64d94d0d5a7640818c21aa5a622624ab => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:27.905557 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e26682-f9e5-4d18-aea8-8b6a7d597075 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:27.967748 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294156af-0e7f-4614-b15b-7b8a08412ae2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:28.016478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abcaa2d-47c9-405f-ae55-6ffc66825f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:28.018630 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcaa2d-47c9-405f-ae55-6ffc66825f3c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:28.019153 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcaa2d-47c9-405f-ae55-6ffc66825f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:28.019446 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcaa2d-47c9-405f-ae55-6ffc66825f3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:28.032364 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abcaa2d-47c9-405f-ae55-6ffc66825f3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:28.033938 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abcaa2d-47c9-405f-ae55-6ffc66825f3c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:28.050440 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:28.054996 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e26682-f9e5-4d18-aea8-8b6a7d597075 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:28.059167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-3abcaa2d-47c9-405f-ae55-6ffc66825f3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:28.059167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:28.059167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:28.059167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:28.059167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:28.059167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:28.059167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:28.059167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:28.059167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:28.059167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:28.059167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:28.059167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:28.059167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:28.059167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:28.059167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:28.060124 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:28.060124 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:28.060124 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:28.060124 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:28.060124 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:28.060124 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:28.060124 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:28.060124 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:28.060124 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:28.060124 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:28.060124 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:28.060124 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:28.060124 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:28.060124 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:28.060124 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:28.060124 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:28.060124 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:28.061111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:28.061111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:28.061111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:28.061111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:28.061111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 302/585] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:28.062110 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abcaa2d-47c9-405f-ae55-6ffc66825f3c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:28.108751 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e58e86-53b6-4051-b253-ae4043926d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:28.109639 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e58e86-53b6-4051-b253-ae4043926d76 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:28.110002 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e58e86-53b6-4051-b253-ae4043926d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:28.110349 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e58e86-53b6-4051-b253-ae4043926d76 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:28.111272 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e58e86-53b6-4051-b253-ae4043926d76 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:28.111680 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e58e86-53b6-4051-b253-ae4043926d76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:28.113902 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e58e86-53b6-4051-b253-ae4043926d76 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:28.145193 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 284/586] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Jul 21 15:51:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:28.145193 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e58e86-53b6-4051-b253-ae4043926d76 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:28.186294 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abcaa2d-47c9-405f-ae55-6ffc66825f3c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:28.204300 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:28.242847 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e7756-0b4c-4f59-8df1-7ed34c79b99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:28.243793 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e7756-0b4c-4f59-8df1-7ed34c79b99c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:28.244269 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e7756-0b4c-4f59-8df1-7ed34c79b99c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:28.244733 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e7756-0b4c-4f59-8df1-7ed34c79b99c None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:28.245694 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034e7756-0b4c-4f59-8df1-7ed34c79b99c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:28.246175 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034e7756-0b4c-4f59-8df1-7ed34c79b99c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:28.247781 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034e7756-0b4c-4f59-8df1-7ed34c79b99c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:28.269626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 303/587] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:28.269626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e7756-0b4c-4f59-8df1-7ed34c79b99c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:28.289168 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e58e86-53b6-4051-b253-ae4043926d76 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:28.337756 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9465110-0b75-4348-ba3f-7d0b59907ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:28.338702 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9465110-0b75-4348-ba3f-7d0b59907ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:28.338992 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9465110-0b75-4348-ba3f-7d0b59907ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:28.339271 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9465110-0b75-4348-ba3f-7d0b59907ef0 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:28.340438 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9465110-0b75-4348-ba3f-7d0b59907ef0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:28.340761 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9465110-0b75-4348-ba3f-7d0b59907ef0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:28.342301 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9465110-0b75-4348-ba3f-7d0b59907ef0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:28.361895 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 285/588] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:28.362981 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9465110-0b75-4348-ba3f-7d0b59907ef0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:28.383964 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:28.402423 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e7756-0b4c-4f59-8df1-7ed34c79b99c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:28.439537 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc166250-326a-42e8-bf9f-725b38431118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:28.440540 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc166250-326a-42e8-bf9f-725b38431118 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:28.440825 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc166250-326a-42e8-bf9f-725b38431118 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:28.441101 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc166250-326a-42e8-bf9f-725b38431118 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:28.452671 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc166250-326a-42e8-bf9f-725b38431118 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:28.455388 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc166250-326a-42e8-bf9f-725b38431118 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:28.473653 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-fc166250-326a-42e8-bf9f-725b38431118 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:28.473653 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:28.473653 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:28.473653 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:28.473653 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:28.473653 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:28.473653 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:28.473653 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:28.473653 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:28.473653 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:28.473653 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:28.473653 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:28.473653 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:28.473653 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:28.473653 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:28.474673 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:28.474673 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:28.474673 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:28.474673 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:28.474673 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:28.474673 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:28.474673 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:28.474673 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:28.474673 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:28.474673 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:28.474673 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:28.474673 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:28.474673 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:28.474673 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:28.474673 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:28.474673 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:28.474673 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:28.475603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:28.475603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:28.475603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:28.475603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:28.479353 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 304/589] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:28.480183 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc166250-326a-42e8-bf9f-725b38431118 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:28.496912 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9465110-0b75-4348-ba3f-7d0b59907ef0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:28.533985 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:28.534600 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3adaf-eeb2-439a-9c76-a5450728a087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:28.535422 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3adaf-eeb2-439a-9c76-a5450728a087 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:28.535705 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3adaf-eeb2-439a-9c76-a5450728a087 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:28.535980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3adaf-eeb2-439a-9c76-a5450728a087 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:28.546945 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d3adaf-eeb2-439a-9c76-a5450728a087 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:28.549076 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d3adaf-eeb2-439a-9c76-a5450728a087 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:28.563371 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-f5d3adaf-eeb2-439a-9c76-a5450728a087 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 15:51:28.563371 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:28.563371 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:28.563371 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:28.563371 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:28.563371 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:28.563371 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:28.563371 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:28.563371 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:28.563371 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:28.563371 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:28.563371 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:28.563371 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:51:28.563371 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:51:28.563371 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:51:28.563371 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:51:28.564419 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:28.564419 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:28.564419 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:51:28.564419 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:28.564419 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:51:28.564419 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:28.564419 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:28.564419 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:28.564419 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:28.564419 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:28.564419 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:28.564419 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:28.564419 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:28.564419 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:28.564419 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:28.564419 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:28.564419 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:28.564419 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:28.565401 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:28.565401 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:28.565401 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:51:28.565401 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:51:28.565401 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:51:28.565401 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:51:28.565401 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:51:28.565401 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:51:28.565401 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 15:51:28.565401 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:28.565401 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 286/590] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:51:28 2022] GET /identity/v3/users/glance => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:28.566508 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3adaf-eeb2-439a-9c76-a5450728a087 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:28.605883 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc166250-326a-42e8-bf9f-725b38431118 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:28.645135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de53ea-139f-4253-8c5a-fa789a64c3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:28.646085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de53ea-139f-4253-8c5a-fa789a64c3ba None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:28.646371 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de53ea-139f-4253-8c5a-fa789a64c3ba None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:28.646646 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de53ea-139f-4253-8c5a-fa789a64c3ba None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:28.648103 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:28.656105 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de53ea-139f-4253-8c5a-fa789a64c3ba None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:28.659085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de53ea-139f-4253-8c5a-fa789a64c3ba None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:28.675875 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d4de53ea-139f-4253-8c5a-fa789a64c3ba None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 15:51:28.675875 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:28.675875 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:28.675875 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:28.675875 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:28.675875 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:28.675875 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:28.675875 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:28.675875 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:28.675875 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:28.675875 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:28.675875 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:28.675875 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:51:28.675875 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:51:28.675875 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:51:28.675875 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:51:28.676925 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:28.676925 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:28.676925 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:51:28.676925 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:28.676925 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:51:28.676925 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:28.676925 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:28.676925 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:28.676925 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:28.676925 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:28.676925 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:28.676925 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:28.676925 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:28.676925 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:28.676925 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:28.676925 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:28.676925 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:28.676925 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:28.677920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:28.677920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:28.677920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:51:28.677920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:51:28.677920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:51:28.677920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:51:28.677920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:51:28.677920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:51:28.677920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 15:51:28.677920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:28.680033 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3adaf-eeb2-439a-9c76-a5450728a087 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:28.685311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 305/591] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 15:51:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:28.686451 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de53ea-139f-4253-8c5a-fa789a64c3ba None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:28.716105 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225aad1-2772-4380-bb5e-14eb2da2dfff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:28.716891 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225aad1-2772-4380-bb5e-14eb2da2dfff None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:28.717241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225aad1-2772-4380-bb5e-14eb2da2dfff None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:28.717657 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225aad1-2772-4380-bb5e-14eb2da2dfff None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:28.718437 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b225aad1-2772-4380-bb5e-14eb2da2dfff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:28.719059 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b225aad1-2772-4380-bb5e-14eb2da2dfff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:28.720706 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b225aad1-2772-4380-bb5e-14eb2da2dfff None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:28.737702 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 287/592] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:28.738742 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225aad1-2772-4380-bb5e-14eb2da2dfff None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:28.780977 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:28.805150 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de53ea-139f-4253-8c5a-fa789a64c3ba None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:28.844599 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225aad1-2772-4380-bb5e-14eb2da2dfff None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:28.845010 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13382d0f-33eb-4be6-9edc-4fc6ba8321d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:28.845936 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13382d0f-33eb-4be6-9edc-4fc6ba8321d0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:28.846491 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13382d0f-33eb-4be6-9edc-4fc6ba8321d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:28.846940 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13382d0f-33eb-4be6-9edc-4fc6ba8321d0 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:28.848381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13382d0f-33eb-4be6-9edc-4fc6ba8321d0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:28.848883 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13382d0f-33eb-4be6-9edc-4fc6ba8321d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:28.850577 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13382d0f-33eb-4be6-9edc-4fc6ba8321d0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:28.870950 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 306/593] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Jul 21 15:51:28 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:28.872080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13382d0f-33eb-4be6-9edc-4fc6ba8321d0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:28.888869 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:28.896041 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aefa4e-2637-44d7-a787-d6c079f862f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:28.896808 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aefa4e-2637-44d7-a787-d6c079f862f1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:28.897158 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aefa4e-2637-44d7-a787-d6c079f862f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:28.897501 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aefa4e-2637-44d7-a787-d6c079f862f1 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:28.898463 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aefa4e-2637-44d7-a787-d6c079f862f1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:28.898841 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aefa4e-2637-44d7-a787-d6c079f862f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:28.900391 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aefa4e-2637-44d7-a787-d6c079f862f1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:28.924289 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 288/594] 10.222.0.52 () {56 vars in 1139 bytes} [Thu Jul 21 15:51:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:28.924289 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aefa4e-2637-44d7-a787-d6c079f862f1 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:28.996137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13382d0f-33eb-4be6-9edc-4fc6ba8321d0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:29.028951 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aefa4e-2637-44d7-a787-d6c079f862f1 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:29.037248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d731d53-a0f6-4881-8051-61e94ba5f5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:29.038505 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d731d53-a0f6-4881-8051-61e94ba5f5ea None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:29.039002 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d731d53-a0f6-4881-8051-61e94ba5f5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:29.039507 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d731d53-a0f6-4881-8051-61e94ba5f5ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:29.047498 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d731d53-a0f6-4881-8051-61e94ba5f5ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:29.049528 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d731d53-a0f6-4881-8051-61e94ba5f5ea None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:29.055955 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:29.066080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-3d731d53-a0f6-4881-8051-61e94ba5f5ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:29.066080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:29.066080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:29.066080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:29.066080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:29.066080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:29.066080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:29.066080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:29.066080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:29.066080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:29.066080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:29.066080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:29.066080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:51:29.066080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:51:29.066080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:51:29.067227 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:51:29.067227 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:29.067227 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:29.067227 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:29.067227 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:29.067227 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:29.067227 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:29.067227 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:29.067227 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:29.067227 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:29.067227 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:29.067227 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:29.067227 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:29.067227 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:29.067227 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:29.067227 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:29.067227 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:29.068397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:51:29.068397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:51:29.068397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:51:29.068397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:51:29.068397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:51:29.068397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:51:29.068397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:29.068397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:29.070621 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 307/595] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:51:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:29.071519 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d731d53-a0f6-4881-8051-61e94ba5f5ea None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:29.091355 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ed876-0296-4c09-be58-12f0178b6f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:29.092343 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ed876-0296-4c09-be58-12f0178b6f9f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:29.092765 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ed876-0296-4c09-be58-12f0178b6f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:29.093134 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ed876-0296-4c09-be58-12f0178b6f9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:29.105295 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ed876-0296-4c09-be58-12f0178b6f9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:29.107089 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ed876-0296-4c09-be58-12f0178b6f9f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:29.127061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-543ed876-0296-4c09-be58-12f0178b6f9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:29.127061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:29.127061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:29.127061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:29.127061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:29.127061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:29.127061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:29.127061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:29.127061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:29.127061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:29.127061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:29.127061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:29.127061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:29.127061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:29.127061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:29.128280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:29.128280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:29.128280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:29.128280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:29.128280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:29.128280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:29.128280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:29.128280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:29.128280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:29.128280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:29.128280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:29.128280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:29.128280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:29.128280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:29.128280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:29.128280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:29.128280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:29.129427 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:29.129427 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:29.129427 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:29.129427 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:29.130918 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 289/596] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:29.131979 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ed876-0296-4c09-be58-12f0178b6f9f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:29.197337 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:29.200136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d731d53-a0f6-4881-8051-61e94ba5f5ea None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:29.248118 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5127eb-005b-44a9-a491-230a02f3f201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:29.249181 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5127eb-005b-44a9-a491-230a02f3f201 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:29.249706 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5127eb-005b-44a9-a491-230a02f3f201 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:29.250241 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5127eb-005b-44a9-a491-230a02f3f201 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:29.262623 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5127eb-005b-44a9-a491-230a02f3f201 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:29.264458 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5127eb-005b-44a9-a491-230a02f3f201 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:29.278409 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ed876-0296-4c09-be58-12f0178b6f9f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:29.284559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-7c5127eb-005b-44a9-a491-230a02f3f201 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:29.284559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:29.284559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:29.284559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:29.284559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:29.284559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:29.284559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:29.284559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:29.284559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:29.284559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:29.284559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:29.284559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:29.284559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:29.284559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:29.284559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:29.285486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:29.285486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:29.285486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:29.285486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:29.285486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:29.285486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:29.285486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:29.285486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:29.285486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:29.285486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:29.285486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:29.285486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:29.285486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:29.285486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:29.285486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:29.285486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:29.285486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:29.286509 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:29.286509 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:29.286509 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:29.286509 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:29.288572 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 308/597] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:29.294046 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5127eb-005b-44a9-a491-230a02f3f201 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:29.322772 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2af51e-2f8e-4c29-b036-62384d8e8231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:29.324019 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2af51e-2f8e-4c29-b036-62384d8e8231 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:29.324581 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2af51e-2f8e-4c29-b036-62384d8e8231 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:29.325262 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2af51e-2f8e-4c29-b036-62384d8e8231 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:29.327028 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2af51e-2f8e-4c29-b036-62384d8e8231 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:29.327902 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2af51e-2f8e-4c29-b036-62384d8e8231 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:29.330904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2af51e-2f8e-4c29-b036-62384d8e8231 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:29.348731 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 290/598] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:51:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:29.349961 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2af51e-2f8e-4c29-b036-62384d8e8231 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:29.359548 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:29.418647 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5127eb-005b-44a9-a491-230a02f3f201 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:29.457778 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9e091c-0ee1-48ae-8343-355db1ff2c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:29.458486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e091c-0ee1-48ae-8343-355db1ff2c67 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:29.458827 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e091c-0ee1-48ae-8343-355db1ff2c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:29.459046 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e091c-0ee1-48ae-8343-355db1ff2c67 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:29.459785 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9e091c-0ee1-48ae-8343-355db1ff2c67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:29.460111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9e091c-0ee1-48ae-8343-355db1ff2c67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:29.461477 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9e091c-0ee1-48ae-8343-355db1ff2c67 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:29.462023 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2af51e-2f8e-4c29-b036-62384d8e8231 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:29.463871 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:29.477085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 309/599] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:29.478028 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9e091c-0ee1-48ae-8343-355db1ff2c67 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:29.504336 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84a644-d740-4ea0-82b9-8dbe7d1e079c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:29.505406 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84a644-d740-4ea0-82b9-8dbe7d1e079c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:29.505926 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84a644-d740-4ea0-82b9-8dbe7d1e079c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:29.506402 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84a644-d740-4ea0-82b9-8dbe7d1e079c None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:29.507406 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf84a644-d740-4ea0-82b9-8dbe7d1e079c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:29.507930 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf84a644-d740-4ea0-82b9-8dbe7d1e079c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:29.509597 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf84a644-d740-4ea0-82b9-8dbe7d1e079c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:29.523482 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 291/600] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:29.524640 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84a644-d740-4ea0-82b9-8dbe7d1e079c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:29.600883 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9e091c-0ee1-48ae-8343-355db1ff2c67 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:29.633154 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:29.645977 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84a644-d740-4ea0-82b9-8dbe7d1e079c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:29.670873 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6187e72-af84-49cb-8b49-b3777e323aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:29.671931 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6187e72-af84-49cb-8b49-b3777e323aac None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:29.672209 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6187e72-af84-49cb-8b49-b3777e323aac None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:29.672491 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6187e72-af84-49cb-8b49-b3777e323aac None admin] PATH_INFO: `/v3/projects/bb1ac00308714c39a79aae09fbb73021/users/87c0598af7714aeba08fffbb6805a49e/roles/64d94d0d5a7640818c21aa5a622624ab` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:29.684840 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d4465-af41-4148-81d0-02cd83a1ce99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:29.686030 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d4465-af41-4148-81d0-02cd83a1ce99 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:29.686536 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d4465-af41-4148-81d0-02cd83a1ce99 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:29.687011 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d4465-af41-4148-81d0-02cd83a1ce99 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:29.691052 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6187e72-af84-49cb-8b49-b3777e323aac None admin] RBAC: Authorizing `identity:create_grant(project_id=bb1ac00308714c39a79aae09fbb73021, user_id=87c0598af7714aeba08fffbb6805a49e, role_id=64d94d0d5a7640818c21aa5a622624ab)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:29.692536 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6187e72-af84-49cb-8b49-b3777e323aac None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:29.695662 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d4465-af41-4148-81d0-02cd83a1ce99 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:29.697453 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d4465-af41-4148-81d0-02cd83a1ce99 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:29.715073 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-a20d4465-af41-4148-81d0-02cd83a1ce99 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:29.715073 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:29.715073 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:29.715073 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:29.715073 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:29.715073 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:29.715073 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:29.715073 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:29.715073 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:29.715073 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:29.715073 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:29.715073 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:29.715073 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:51:29.715073 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:51:29.715073 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:51:29.716080 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:51:29.716080 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:29.716080 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:29.716080 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:29.716080 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:29.716080 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:29.716080 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:29.716080 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:29.716080 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:29.716080 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:29.716080 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:29.716080 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:29.716080 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:29.716080 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:29.716080 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:29.716080 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:29.716080 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:29.716991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:51:29.716991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:51:29.716991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:51:29.716991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:51:29.716991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:51:29.716991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:51:29.716991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:29.716991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:29.717962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 310/601] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 15:51:29 2022] PUT /identity/v3/projects/bb1ac00308714c39a79aae09fbb73021/users/87c0598af7714aeba08fffbb6805a49e/roles/64d94d0d5a7640818c21aa5a622624ab => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 15:51:29.718814 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6187e72-af84-49cb-8b49-b3777e323aac None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:29.721120 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 292/602] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:51:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:29.729670 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d4465-af41-4148-81d0-02cd83a1ce99 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:29.774894 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:29.871704 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6187e72-af84-49cb-8b49-b3777e323aac None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:29.926684 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88819818-8a50-4439-8fce-551e772d0590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:29.927975 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88819818-8a50-4439-8fce-551e772d0590 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:29.928489 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88819818-8a50-4439-8fce-551e772d0590 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:29.929083 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88819818-8a50-4439-8fce-551e772d0590 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:29.943230 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88819818-8a50-4439-8fce-551e772d0590 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:29.946583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88819818-8a50-4439-8fce-551e772d0590 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:29.967552 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-88819818-8a50-4439-8fce-551e772d0590 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:29.967552 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:29.967552 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:29.967552 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:29.967552 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:29.967552 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:29.967552 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:29.967552 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:29.967552 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:29.967552 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:29.967552 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:29.967552 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:29.967552 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:51:29.967552 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:51:29.967552 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:51:29.969674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:51:29.969674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:29.969674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:29.969674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:29.969674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:29.969674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:29.969674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:29.969674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:29.969674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:29.969674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:29.969674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:29.969674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:29.969674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:29.969674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:29.969674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:29.969674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:29.969674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:29.970735 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:51:29.970735 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:51:29.970735 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:51:29.970735 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:51:29.970735 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:51:29.970735 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:51:29.970735 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:29.970735 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:29.971195 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d4465-af41-4148-81d0-02cd83a1ce99 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:29.973337 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 311/603] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:51:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:29.986668 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88819818-8a50-4439-8fce-551e772d0590 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:30.012629 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:30.022435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75591c1-850c-4d8d-84d6-0dc3fa97c101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:30.023357 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75591c1-850c-4d8d-84d6-0dc3fa97c101 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:30.023474 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75591c1-850c-4d8d-84d6-0dc3fa97c101 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:30.023770 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75591c1-850c-4d8d-84d6-0dc3fa97c101 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:30.024652 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75591c1-850c-4d8d-84d6-0dc3fa97c101 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:30.024971 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75591c1-850c-4d8d-84d6-0dc3fa97c101 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:30.026526 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75591c1-850c-4d8d-84d6-0dc3fa97c101 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:30.043415 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 293/604] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:51:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:30.054248 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75591c1-850c-4d8d-84d6-0dc3fa97c101 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:30.114865 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88819818-8a50-4439-8fce-551e772d0590 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:30.154180 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:30.165962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a834bda-567d-4c93-8a9d-d34aab9ac090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:30.167342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a834bda-567d-4c93-8a9d-d34aab9ac090 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:30.168000 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a834bda-567d-4c93-8a9d-d34aab9ac090 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:30.168543 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a834bda-567d-4c93-8a9d-d34aab9ac090 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:30.170028 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a834bda-567d-4c93-8a9d-d34aab9ac090 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:30.170472 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a834bda-567d-4c93-8a9d-d34aab9ac090 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:30.174261 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a834bda-567d-4c93-8a9d-d34aab9ac090 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:30.176218 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75591c1-850c-4d8d-84d6-0dc3fa97c101 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:30.187346 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 312/605] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:51:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:30.191469 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d7eedd-f934-49dc-b854-11640dfd2d83 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:30.191469 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d7eedd-f934-49dc-b854-11640dfd2d83 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:30.191469 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d7eedd-f934-49dc-b854-11640dfd2d83 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:30.191469 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 313/606] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:30.191947 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c790fd-a745-4897-b1fd-9b752e953eef None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:30.192173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c790fd-a745-4897-b1fd-9b752e953eef None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:30.192467 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c790fd-a745-4897-b1fd-9b752e953eef None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:30.194626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 314/607] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:30.195411 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c790fd-a745-4897-b1fd-9b752e953eef None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:30.231841 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79467162-f239-4048-ac01-8bc387ec7422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:30.233147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79467162-f239-4048-ac01-8bc387ec7422 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:30.233441 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79467162-f239-4048-ac01-8bc387ec7422 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:30.233882 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79467162-f239-4048-ac01-8bc387ec7422 None admin] PATH_INFO: `/v3/projects/bb1ac00308714c39a79aae09fbb73021/users/d73a7d5f37e144768d7cc53f13cd5157/roles/64d94d0d5a7640818c21aa5a622624ab` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:30.251780 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79467162-f239-4048-ac01-8bc387ec7422 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb1ac00308714c39a79aae09fbb73021, user_id=d73a7d5f37e144768d7cc53f13cd5157, role_id=64d94d0d5a7640818c21aa5a622624ab)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:30.253699 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79467162-f239-4048-ac01-8bc387ec7422 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:30.274236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 294/608] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 15:51:30 2022] PUT /identity/v3/projects/bb1ac00308714c39a79aae09fbb73021/users/d73a7d5f37e144768d7cc53f13cd5157/roles/64d94d0d5a7640818c21aa5a622624ab => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 15:51:30.275919 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5212d6b-dee9-4a59-b366-4994951685d3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:30.276449 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5212d6b-dee9-4a59-b366-4994951685d3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:30.276927 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5212d6b-dee9-4a59-b366-4994951685d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:30.295875 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d5212d6b-dee9-4a59-b366-4994951685d3 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:30.298096 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d5212d6b-dee9-4a59-b366-4994951685d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:30.346101 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:30.365282 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c790fd-a745-4897-b1fd-9b752e953eef None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:30.401627 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 295/609] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:51:30.401627 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f79c9-0c27-41f5-be69-79ba66b7ee0d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:30.401627 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f79c9-0c27-41f5-be69-79ba66b7ee0d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:30.401627 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f79c9-0c27-41f5-be69-79ba66b7ee0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:30.427086 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5b9f79c9-0c27-41f5-be69-79ba66b7ee0d None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:30.429648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5b9f79c9-0c27-41f5-be69-79ba66b7ee0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:30.434808 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd69ba1-648f-4120-b93f-41e5ea07c625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:30.435735 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd69ba1-648f-4120-b93f-41e5ea07c625 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:30.436019 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd69ba1-648f-4120-b93f-41e5ea07c625 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:30.436297 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd69ba1-648f-4120-b93f-41e5ea07c625 None admin] PATH_INFO: `/v3/projects/bb1ac00308714c39a79aae09fbb73021/users/1654fdf5ba6b46ba8d9b86a83a76a4b5/roles/64d94d0d5a7640818c21aa5a622624ab` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:30.457155 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd69ba1-648f-4120-b93f-41e5ea07c625 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb1ac00308714c39a79aae09fbb73021, user_id=1654fdf5ba6b46ba8d9b86a83a76a4b5, role_id=64d94d0d5a7640818c21aa5a622624ab)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:30.458687 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd69ba1-648f-4120-b93f-41e5ea07c625 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:30.480744 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 315/610] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 15:51:30 2022] PUT /identity/v3/projects/bb1ac00308714c39a79aae09fbb73021/users/1654fdf5ba6b46ba8d9b86a83a76a4b5/roles/64d94d0d5a7640818c21aa5a622624ab => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 15:51:30.481744 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd69ba1-648f-4120-b93f-41e5ea07c625 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:30.524134 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:30.550107 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 296/611] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:51:30.589528 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f79c9-0c27-41f5-be69-79ba66b7ee0d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:30.680008 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd69ba1-648f-4120-b93f-41e5ea07c625 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:30.717206 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:30.725814 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f542d8b-e505-4f85-915f-89c9dbf13cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:30.726645 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f542d8b-e505-4f85-915f-89c9dbf13cbc None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:30.727237 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f542d8b-e505-4f85-915f-89c9dbf13cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:30.727765 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f542d8b-e505-4f85-915f-89c9dbf13cbc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:30.743645 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f542d8b-e505-4f85-915f-89c9dbf13cbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:30.752173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-9f542d8b-e505-4f85-915f-89c9dbf13cbc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:30.752173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:30.752173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:30.752173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:30.752173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:30.752173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:30.752173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:30.752173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:30.752173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:30.752173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:30.752173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:30.752173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:30.752173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:30.752173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:30.752173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:30.752173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:30.753237 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:30.753237 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:30.753237 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:30.753237 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:30.753237 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:30.753237 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:30.753237 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:30.753237 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:30.753237 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:30.753237 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:30.753237 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:30.753237 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:30.753237 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:30.753237 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:30.753237 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:30.753237 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:30.754187 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:30.754187 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:30.754187 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:30.754187 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:30.754187 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:30.754187 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:30.754187 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:30.754187 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:30.754187 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:30.754187 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:30.754187 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:30.754187 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:30.754187 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 316/612] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 15:51:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:30.763481 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f542d8b-e505-4f85-915f-89c9dbf13cbc None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:30.786449 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f79c9-0c27-41f5-be69-79ba66b7ee0d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:30.849300 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af22cc7-5269-4dac-8dc0-017dd48c7422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:30.850139 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af22cc7-5269-4dac-8dc0-017dd48c7422 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:30.850426 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af22cc7-5269-4dac-8dc0-017dd48c7422 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:30.850707 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af22cc7-5269-4dac-8dc0-017dd48c7422 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:30.873029 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af22cc7-5269-4dac-8dc0-017dd48c7422 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:30.889246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-1af22cc7-5269-4dac-8dc0-017dd48c7422 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:30.889246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:30.889246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:30.889246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:30.889246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:30.889246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:30.889246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:30.889246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:30.889246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:30.889246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:30.889246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:30.889246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:30.889246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:30.889246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:30.889246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:30.889246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:30.891586 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:30.891586 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:30.891586 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:30.891586 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:30.891586 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:30.891586 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:30.891586 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:30.891586 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:30.891586 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:30.891586 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:30.891586 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:30.891586 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:30.891586 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:30.891586 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:30.891586 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:30.891586 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:30.893141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:30.893141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:30.893141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:30.893141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:30.893141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:30.893141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:30.893141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:30.893141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:30.893141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:30.893141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:30.893141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:30.893141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:30.902052 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 297/613] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 15:51:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:51:30.910660 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af22cc7-5269-4dac-8dc0-017dd48c7422 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:30.978238 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:31.028643 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af22cc7-5269-4dac-8dc0-017dd48c7422 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:31.067873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ea21c5-74bb-4d60-816e-4e03bd68b12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:31.068584 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f542d8b-e505-4f85-915f-89c9dbf13cbc None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:31.069147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea21c5-74bb-4d60-816e-4e03bd68b12e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:31.069720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea21c5-74bb-4d60-816e-4e03bd68b12e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:31.070414 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea21c5-74bb-4d60-816e-4e03bd68b12e None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:31.071535 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ea21c5-74bb-4d60-816e-4e03bd68b12e None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:31.072067 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ea21c5-74bb-4d60-816e-4e03bd68b12e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:31.073735 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ea21c5-74bb-4d60-816e-4e03bd68b12e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:31.085849 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 298/614] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 15:51:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:51:31.091682 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ea21c5-74bb-4d60-816e-4e03bd68b12e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:31.151573 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8accf-7b9f-4cb4-b8d3-ae6ed9b22577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:31.152768 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8accf-7b9f-4cb4-b8d3-ae6ed9b22577 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:31.154698 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8accf-7b9f-4cb4-b8d3-ae6ed9b22577 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:31.155214 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8accf-7b9f-4cb4-b8d3-ae6ed9b22577 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:31.156209 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f8accf-7b9f-4cb4-b8d3-ae6ed9b22577 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:31.158170 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f8accf-7b9f-4cb4-b8d3-ae6ed9b22577 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:31.160828 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f8accf-7b9f-4cb4-b8d3-ae6ed9b22577 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:31.182698 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 317/615] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 15:51:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:51:31.188789 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8accf-7b9f-4cb4-b8d3-ae6ed9b22577 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:31.269920 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ea21c5-74bb-4d60-816e-4e03bd68b12e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:31.301959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:31.310359 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:31.330947 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b1625-e7c8-4676-9437-ff20a743748f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:31.332180 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b1625-e7c8-4676-9437-ff20a743748f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:31.332688 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b1625-e7c8-4676-9437-ff20a743748f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:31.333219 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b1625-e7c8-4676-9437-ff20a743748f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:31.345677 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b1625-e7c8-4676-9437-ff20a743748f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:31.347811 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b1625-e7c8-4676-9437-ff20a743748f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:31.366339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-bc0b1625-e7c8-4676-9437-ff20a743748f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:31.366339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:31.366339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:31.366339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:31.366339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:31.366339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:31.366339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:31.366339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:31.366339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:31.366339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:31.366339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:31.366339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:31.366339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:31.366339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:31.366339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:31.367337 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:31.367337 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:31.367337 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:31.367337 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:31.367337 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:31.367337 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:31.367337 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:31.367337 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:31.367337 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:31.367337 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:31.367337 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:31.367337 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:31.367337 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:31.367337 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:31.367337 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:31.367337 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:31.367337 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:31.368284 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:31.368284 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:31.368284 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:31.368284 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:31.370596 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 299/616] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:51:31.376907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b1625-e7c8-4676-9437-ff20a743748f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:31.447839 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8accf-7b9f-4cb4-b8d3-ae6ed9b22577 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:31.487044 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b1625-e7c8-4676-9437-ff20a743748f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:31.523755 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643356d-5bd8-495a-a597-f5b07d4c1baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:31.524747 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643356d-5bd8-495a-a597-f5b07d4c1baf None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:31.525232 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643356d-5bd8-495a-a597-f5b07d4c1baf None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:31.525767 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643356d-5bd8-495a-a597-f5b07d4c1baf None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:31.526916 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643356d-5bd8-495a-a597-f5b07d4c1baf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:31.527424 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643356d-5bd8-495a-a597-f5b07d4c1baf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:31.528996 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643356d-5bd8-495a-a597-f5b07d4c1baf None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:31.534277 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec8c093-03f7-4fc7-a3d0-cfd2a970a81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:31.535093 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8c093-03f7-4fc7-a3d0-cfd2a970a81a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:31.535385 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8c093-03f7-4fc7-a3d0-cfd2a970a81a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:31.535667 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8c093-03f7-4fc7-a3d0-cfd2a970a81a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:31.544218 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 300/617] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:51:31.550019 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643356d-5bd8-495a-a597-f5b07d4c1baf None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:31.555745 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec8c093-03f7-4fc7-a3d0-cfd2a970a81a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:31.557227 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec8c093-03f7-4fc7-a3d0-cfd2a970a81a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:31.582165 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:31.604570 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-7ec8c093-03f7-4fc7-a3d0-cfd2a970a81a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:31.604570 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:31.604570 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:31.604570 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:31.604570 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:31.604570 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:31.604570 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:31.604570 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:31.604570 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:31.604570 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:31.604570 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:31.604570 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:31.604570 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:31.604570 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:31.604570 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:31.605566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:31.605566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:31.605566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:31.605566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:31.605566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:31.605566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:31.605566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:31.605566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:31.605566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:31.605566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:31.605566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:31.605566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:31.605566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:31.605566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:31.605566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:31.605566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:31.605566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:31.606518 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:31.606518 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:31.606518 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:31.606518 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:31.607174 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 318/618] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 418 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:51:31.614998 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec8c093-03f7-4fc7-a3d0-cfd2a970a81a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:31.743862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643356d-5bd8-495a-a597-f5b07d4c1baf None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:31.744601 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec8c093-03f7-4fc7-a3d0-cfd2a970a81a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:31.747181 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:31.783761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a35d2-e0e5-494f-9e43-58011fa5dc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:31.784362 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a35d2-e0e5-494f-9e43-58011fa5dc35 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:31.784644 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a35d2-e0e5-494f-9e43-58011fa5dc35 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:31.784923 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a35d2-e0e5-494f-9e43-58011fa5dc35 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:31.785851 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a35d2-e0e5-494f-9e43-58011fa5dc35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:31.786196 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a35d2-e0e5-494f-9e43-58011fa5dc35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:31.787597 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a35d2-e0e5-494f-9e43-58011fa5dc35 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:31.800901 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 319/619] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:31.806283 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a35d2-e0e5-494f-9e43-58011fa5dc35 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:31.850853 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2fdc42-ebb5-487f-ab77-c08938a45216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:31.851690 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2fdc42-ebb5-487f-ab77-c08938a45216 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:31.851980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2fdc42-ebb5-487f-ab77-c08938a45216 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:31.852258 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2fdc42-ebb5-487f-ab77-c08938a45216 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:31.874517 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2fdc42-ebb5-487f-ab77-c08938a45216 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:31.876417 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2fdc42-ebb5-487f-ab77-c08938a45216 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:31.916285 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-8e2fdc42-ebb5-487f-ab77-c08938a45216 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 15:51:31.916285 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:31.916285 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:31.916285 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:31.916285 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:31.916285 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:31.916285 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:31.916285 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:31.916285 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:31.916285 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:31.916285 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:31.916285 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:31.916285 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:51:31.916285 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:51:31.916285 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:51:31.916285 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:51:31.917653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:31.917653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:31.917653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:51:31.917653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:31.917653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:51:31.917653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:31.917653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:31.917653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:31.917653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:31.917653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:31.917653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:31.917653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:31.917653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:31.917653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:31.917653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:31.917653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:31.917653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:31.917653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:31.919136 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:31.919136 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:31.919136 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:51:31.919136 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:51:31.919136 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:51:31.919136 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:51:31.919136 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:51:31.919136 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:51:31.919136 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 15:51:31.919136 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:31.933620 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 301/620] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jul 21 15:51:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:51:31.937630 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2fdc42-ebb5-487f-ab77-c08938a45216 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:31.956145 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a35d2-e0e5-494f-9e43-58011fa5dc35 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:32.036656 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd016ee4-9256-496c-bbf7-cfe2fe11a5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:32.037376 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd016ee4-9256-496c-bbf7-cfe2fe11a5dd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:32.037682 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd016ee4-9256-496c-bbf7-cfe2fe11a5dd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:32.037960 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd016ee4-9256-496c-bbf7-cfe2fe11a5dd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:32.051977 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd016ee4-9256-496c-bbf7-cfe2fe11a5dd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:32.053559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd016ee4-9256-496c-bbf7-cfe2fe11a5dd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:32.071342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-dd016ee4-9256-496c-bbf7-cfe2fe11a5dd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 15:51:32.071342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:32.071342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:32.071342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:32.071342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:32.071342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:32.071342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:32.071342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:32.071342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:32.071342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:32.071342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:32.071342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:32.071342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:51:32.071342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:51:32.071342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:51:32.071342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:51:32.072356 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:32.072356 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:32.072356 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:51:32.072356 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:32.072356 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:51:32.072356 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:32.072356 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:32.072356 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:32.072356 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:32.072356 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:32.072356 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:32.072356 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:32.072356 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:32.072356 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:32.072356 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:32.072356 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:32.072356 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:32.072356 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:32.073345 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:32.073345 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:32.073345 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:51:32.073345 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:51:32.073345 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:51:32.073345 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:51:32.073345 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:51:32.073345 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:51:32.073345 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 15:51:32.073345 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:32.078945 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 320/621] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:51:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:51:32.078945 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd016ee4-9256-496c-bbf7-cfe2fe11a5dd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:32.117988 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:32.145958 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:32.165216 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2fdc42-ebb5-487f-ab77-c08938a45216 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:32.224870 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d9fe4-30d9-48af-b51e-34f2513eb5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:32.225557 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d9fe4-30d9-48af-b51e-34f2513eb5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:32.225864 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d9fe4-30d9-48af-b51e-34f2513eb5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:32.226139 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d9fe4-30d9-48af-b51e-34f2513eb5e8 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:32.226991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d9fe4-30d9-48af-b51e-34f2513eb5e8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:32.227302 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d9fe4-30d9-48af-b51e-34f2513eb5e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:32.228755 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d9fe4-30d9-48af-b51e-34f2513eb5e8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:32.247931 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd016ee4-9256-496c-bbf7-cfe2fe11a5dd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:32.272107 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 302/622] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jul 21 15:51:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:32.277982 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d9fe4-30d9-48af-b51e-34f2513eb5e8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:32.341943 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391148af-18fe-4e14-888c-366500552f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:32.342549 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391148af-18fe-4e14-888c-366500552f6e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:32.342882 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391148af-18fe-4e14-888c-366500552f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:32.343166 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391148af-18fe-4e14-888c-366500552f6e None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:32.344517 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391148af-18fe-4e14-888c-366500552f6e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:32.344836 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391148af-18fe-4e14-888c-366500552f6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:32.347506 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391148af-18fe-4e14-888c-366500552f6e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:32.382014 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 321/623] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Jul 21 15:51:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:32.404435 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391148af-18fe-4e14-888c-366500552f6e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:32.443640 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d9fe4-30d9-48af-b51e-34f2513eb5e8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:32.485734 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57f4fc3-0116-47cf-9b1a-0645d5c2d18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:32.486459 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f4fc3-0116-47cf-9b1a-0645d5c2d18e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:32.486874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f4fc3-0116-47cf-9b1a-0645d5c2d18e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:32.487287 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f4fc3-0116-47cf-9b1a-0645d5c2d18e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:32.510013 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:32.568846 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57f4fc3-0116-47cf-9b1a-0645d5c2d18e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:32.570849 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57f4fc3-0116-47cf-9b1a-0645d5c2d18e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:32.618151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:32.644503 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391148af-18fe-4e14-888c-366500552f6e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:32.694722 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9096f4-a209-4cd4-94c3-064c0f83c6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:32.695657 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9096f4-a209-4cd4-94c3-064c0f83c6ce None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:32.696006 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9096f4-a209-4cd4-94c3-064c0f83c6ce None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:32.696361 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9096f4-a209-4cd4-94c3-064c0f83c6ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:32.711953 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 303/624] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:51:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 15:51:32.712224 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9096f4-a209-4cd4-94c3-064c0f83c6ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:32.713467 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720febb9-5e22-4be4-b14f-4e2045173238 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:32.713791 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720febb9-5e22-4be4-b14f-4e2045173238 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:32.714034 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9096f4-a209-4cd4-94c3-064c0f83c6ce None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:32.714162 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720febb9-5e22-4be4-b14f-4e2045173238 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:32.715689 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 304/625] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:32.717072 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720febb9-5e22-4be4-b14f-4e2045173238 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:32.739438 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-cb9096f4-a209-4cd4-94c3-064c0f83c6ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:32.739438 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:32.739438 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:32.739438 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:32.739438 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:32.739438 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:32.739438 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:32.739438 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:32.739438 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:32.739438 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:32.739438 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:32.739438 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:32.739438 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:32.739438 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:32.739438 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:32.740630 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:32.740630 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:32.740630 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:32.740630 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:32.740630 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:32.740630 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:32.740630 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:32.740630 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:32.740630 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:32.740630 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:32.740630 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:32.740630 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:32.740630 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:32.740630 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:32.740630 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:32.740630 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:32.740630 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:32.741803 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:32.741803 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:32.741803 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:32.741803 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:32.745616 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 322/626] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:32.745616 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf96aa-2a5a-4d1a-85d5-942a9a81c7a1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:32.745616 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf96aa-2a5a-4d1a-85d5-942a9a81c7a1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:32.745616 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf96aa-2a5a-4d1a-85d5-942a9a81c7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:32.761954 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fcaf96aa-2a5a-4d1a-85d5-942a9a81c7a1 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:32.763345 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fcaf96aa-2a5a-4d1a-85d5-942a9a81c7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:32.785862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:32.842114 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 323/627] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:51:32.843116 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf96aa-2a5a-4d1a-85d5-942a9a81c7a1 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:32.866457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720febb9-5e22-4be4-b14f-4e2045173238 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:32.910618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f08f7b-13a8-4fea-9514-cb8649f85d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:32.911344 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f08f7b-13a8-4fea-9514-cb8649f85d92 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:32.912151 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f08f7b-13a8-4fea-9514-cb8649f85d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:32.912442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f08f7b-13a8-4fea-9514-cb8649f85d92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:32.931256 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f08f7b-13a8-4fea-9514-cb8649f85d92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:32.932748 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f08f7b-13a8-4fea-9514-cb8649f85d92 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:32.943036 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:32.952347 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-97f08f7b-13a8-4fea-9514-cb8649f85d92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:32.952347 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:32.952347 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:32.952347 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:32.952347 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:32.952347 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:32.952347 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:32.952347 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:32.952347 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:32.952347 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:32.952347 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:32.952347 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:32.952347 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:32.952347 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:32.952347 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:32.953943 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:32.953943 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:32.953943 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:32.953943 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:32.953943 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:32.953943 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:32.953943 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:32.953943 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:32.953943 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:32.953943 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:32.953943 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:32.953943 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:32.953943 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:32.953943 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:32.953943 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:32.953943 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:32.953943 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:32.955063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:32.955063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:32.955063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:32.955063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:32.958652 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 305/628] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:32.960046 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe7eb0-1f64-4783-8e0f-dd7b03298b6a None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:32.960340 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe7eb0-1f64-4783-8e0f-dd7b03298b6a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:32.960629 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe7eb0-1f64-4783-8e0f-dd7b03298b6a None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:32.962018 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 306/629] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:32.962738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fe7eb0-1f64-4783-8e0f-dd7b03298b6a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:33.006796 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf96aa-2a5a-4d1a-85d5-942a9a81c7a1 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:33.046579 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a185b1-ae21-4998-ac3c-8f3c73e57d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:33.047250 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a185b1-ae21-4998-ac3c-8f3c73e57d93 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:33.047531 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a185b1-ae21-4998-ac3c-8f3c73e57d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:33.047802 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a185b1-ae21-4998-ac3c-8f3c73e57d93 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:33.048531 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a185b1-ae21-4998-ac3c-8f3c73e57d93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:33.048837 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a185b1-ae21-4998-ac3c-8f3c73e57d93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:33.050248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a185b1-ae21-4998-ac3c-8f3c73e57d93 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:33.062406 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 324/630] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:33.063739 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a185b1-ae21-4998-ac3c-8f3c73e57d93 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:33.080088 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:33.087799 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fe7eb0-1f64-4783-8e0f-dd7b03298b6a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:33.123322 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c407b3-141b-4562-8840-7d4dd181e372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:33.124134 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c407b3-141b-4562-8840-7d4dd181e372 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:33.124417 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c407b3-141b-4562-8840-7d4dd181e372 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:33.124692 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c407b3-141b-4562-8840-7d4dd181e372 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:33.135626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c407b3-141b-4562-8840-7d4dd181e372 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:33.143320 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-f5c407b3-141b-4562-8840-7d4dd181e372 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:33.143320 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:33.143320 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:33.143320 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:33.143320 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:33.143320 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:33.143320 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:33.143320 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:33.143320 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:33.143320 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:33.143320 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:33.143320 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:33.143320 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:33.143320 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:33.143320 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:33.143320 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:33.144480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:33.144480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:33.144480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:33.144480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:33.144480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:33.144480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:33.144480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:33.144480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:33.144480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:33.144480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:33.144480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:33.144480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:33.144480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:33.144480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:33.144480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:33.144480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:33.145539 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:33.145539 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:33.145539 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:33.145539 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:33.145539 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:33.145539 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:33.145539 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:33.145539 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:33.145539 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:33.145539 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:33.145539 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:33.145539 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:33.145539 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 307/631] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 15:51:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:33.146459 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b134e20-1dea-48a2-8d42-61e4819143aa None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:33.146459 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b134e20-1dea-48a2-8d42-61e4819143aa None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:33.146459 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b134e20-1dea-48a2-8d42-61e4819143aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:33.163426 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8b134e20-1dea-48a2-8d42-61e4819143aa None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:33.164561 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8b134e20-1dea-48a2-8d42-61e4819143aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:33.191274 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a185b1-ae21-4998-ac3c-8f3c73e57d93 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:33.215544 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:33.226765 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f410f0-7ea8-46ec-b82d-7eceeafe46a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:33.227437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f410f0-7ea8-46ec-b82d-7eceeafe46a4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:33.227724 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f410f0-7ea8-46ec-b82d-7eceeafe46a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:33.228008 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f410f0-7ea8-46ec-b82d-7eceeafe46a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:33.228744 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f410f0-7ea8-46ec-b82d-7eceeafe46a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:33.229057 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f410f0-7ea8-46ec-b82d-7eceeafe46a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:33.230489 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f410f0-7ea8-46ec-b82d-7eceeafe46a4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:33.243434 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 325/632] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:33.244782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbcf86-290e-4e0a-9ede-aa1a8956deec None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:33.245143 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbcf86-290e-4e0a-9ede-aa1a8956deec None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:33.245352 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbcf86-290e-4e0a-9ede-aa1a8956deec None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:33.246622 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 326/633] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:33.247321 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bbcf86-290e-4e0a-9ede-aa1a8956deec None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:33.261957 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 308/634] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:51:33.262659 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b134e20-1dea-48a2-8d42-61e4819143aa None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:33.353714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:33.374394 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b134e20-1dea-48a2-8d42-61e4819143aa None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:33.384303 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bbcf86-290e-4e0a-9ede-aa1a8956deec None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:33.410689 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6628f2d-7242-440b-850a-1dcf5589ef22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:33.411367 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6628f2d-7242-440b-850a-1dcf5589ef22 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:33.411654 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6628f2d-7242-440b-850a-1dcf5589ef22 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:33.411933 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6628f2d-7242-440b-850a-1dcf5589ef22 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:33.412703 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6628f2d-7242-440b-850a-1dcf5589ef22 None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:33.413015 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6628f2d-7242-440b-850a-1dcf5589ef22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:33.414524 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6628f2d-7242-440b-850a-1dcf5589ef22 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:33.418449 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83313c71-eaa5-4942-978d-52f7122bd141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:33.419237 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83313c71-eaa5-4942-978d-52f7122bd141 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:33.419517 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83313c71-eaa5-4942-978d-52f7122bd141 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:33.419789 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83313c71-eaa5-4942-978d-52f7122bd141 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:33.423963 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 309/635] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 15:51:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:33.424809 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6628f2d-7242-440b-850a-1dcf5589ef22 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:33.427676 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83313c71-eaa5-4942-978d-52f7122bd141 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:33.429154 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83313c71-eaa5-4942-978d-52f7122bd141 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:33.443952 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-83313c71-eaa5-4942-978d-52f7122bd141 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:33.443952 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:33.443952 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:33.443952 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:33.443952 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:33.443952 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:33.443952 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:33.443952 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:33.443952 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:33.443952 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:33.443952 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:33.443952 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:33.443952 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:51:33.443952 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:51:33.443952 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:51:33.445137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:51:33.445137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:33.445137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:33.445137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:33.445137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:33.445137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:33.445137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:33.445137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:33.445137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:33.445137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:33.445137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:33.445137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:33.445137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:33.445137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:33.445137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:33.445137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:33.445137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:33.446304 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:51:33.446304 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:51:33.446304 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:51:33.446304 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:51:33.446304 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:51:33.446304 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:51:33.446304 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:33.446304 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:33.446304 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 327/636] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:51:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:33.446922 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8443bf0-c57c-4d2a-95ca-c1787a629b42 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:33.446922 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8443bf0-c57c-4d2a-95ca-c1787a629b42 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:33.446922 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8443bf0-c57c-4d2a-95ca-c1787a629b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:33.463265 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c8443bf0-c57c-4d2a-95ca-c1787a629b42 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:33.464476 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c8443bf0-c57c-4d2a-95ca-c1787a629b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:33.510714 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:33.534848 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6628f2d-7242-440b-850a-1dcf5589ef22 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:33.541763 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 328/637] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:51:33.542782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8443bf0-c57c-4d2a-95ca-c1787a629b42 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:33.576156 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95874dd6-8944-42ea-949a-87f3b161d175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:33.576950 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95874dd6-8944-42ea-949a-87f3b161d175 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:33.577232 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95874dd6-8944-42ea-949a-87f3b161d175 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:33.577510 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95874dd6-8944-42ea-949a-87f3b161d175 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:33.584499 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95874dd6-8944-42ea-949a-87f3b161d175 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:33.586000 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95874dd6-8944-42ea-949a-87f3b161d175 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:33.601155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-95874dd6-8944-42ea-949a-87f3b161d175 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:33.601155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:33.601155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:33.601155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:33.601155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:33.601155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:33.601155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:33.601155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:33.601155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:33.601155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:33.601155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:33.601155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:33.601155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:51:33.601155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:51:33.601155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:51:33.602187 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:51:33.602187 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:33.602187 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:33.602187 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:33.602187 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:33.602187 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:33.602187 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:33.602187 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:33.602187 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:33.602187 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:33.602187 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:33.602187 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:33.602187 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:33.602187 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:33.602187 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:33.602187 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:33.602187 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:33.603648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:51:33.603648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:51:33.603648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:51:33.603648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:51:33.603648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:51:33.603648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:51:33.603648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:33.603648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:33.603648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 310/638] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:51:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:33.603648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95874dd6-8944-42ea-949a-87f3b161d175 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:33.617417 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:33.659307 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8443bf0-c57c-4d2a-95ca-c1787a629b42 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:33.700434 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ed11d-def3-4958-8c3c-41aa2ce782c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:33.701163 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ed11d-def3-4958-8c3c-41aa2ce782c2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:33.701446 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ed11d-def3-4958-8c3c-41aa2ce782c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:33.701755 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ed11d-def3-4958-8c3c-41aa2ce782c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:33.713429 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ed11d-def3-4958-8c3c-41aa2ce782c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:33.716227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95874dd6-8944-42ea-949a-87f3b161d175 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:33.721338 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-1a0ed11d-def3-4958-8c3c-41aa2ce782c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:33.721338 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:33.721338 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:33.721338 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:33.721338 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:33.721338 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:33.721338 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:33.721338 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:33.721338 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:33.721338 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:33.721338 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:33.721338 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:33.721338 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:33.721338 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:33.721338 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:33.721338 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:33.722430 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:33.722430 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:33.722430 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:33.722430 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:33.722430 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:33.722430 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:33.722430 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:33.722430 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:33.722430 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:33.722430 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:33.722430 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:33.722430 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:33.722430 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:33.722430 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:33.722430 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:33.722430 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:33.723559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:33.723559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:33.723559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:33.723559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:33.723559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:33.723559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:33.723559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:33.723559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:33.723559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:33.723559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:33.723559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:33.723559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:33.723559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 329/639] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 15:51:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:33.724274 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ed11d-def3-4958-8c3c-41aa2ce782c2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:33.759859 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d057c-8c36-4645-8e49-266cbdc0c1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:33.760743 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d057c-8c36-4645-8e49-266cbdc0c1c1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:33.760947 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d057c-8c36-4645-8e49-266cbdc0c1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:33.761333 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d057c-8c36-4645-8e49-266cbdc0c1c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:33.771930 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:33.773302 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d057c-8c36-4645-8e49-266cbdc0c1c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:33.774808 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d057c-8c36-4645-8e49-266cbdc0c1c1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:33.801768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ab4d057c-8c36-4645-8e49-266cbdc0c1c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:33.801768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:33.801768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:33.801768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:33.801768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:33.801768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:33.801768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:33.801768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:33.801768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:33.801768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:33.801768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:33.801768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:33.801768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:33.801768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:33.801768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:33.802851 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:33.802851 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:33.802851 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:33.802851 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:33.802851 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:33.802851 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:33.802851 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:33.802851 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:33.802851 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:33.802851 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:33.802851 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:33.802851 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:33.802851 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:33.802851 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:33.802851 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:33.802851 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:33.802851 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:33.803794 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:33.803794 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:33.803794 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:33.803794 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:33.803794 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 311/640] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:33.804090 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d057c-8c36-4645-8e49-266cbdc0c1c1 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:33.850545 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ed11d-def3-4958-8c3c-41aa2ce782c2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:33.866422 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:33.885459 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b8177-c4c2-454e-8066-4a2da142b176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:33.886163 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b8177-c4c2-454e-8066-4a2da142b176 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:33.886444 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b8177-c4c2-454e-8066-4a2da142b176 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:33.886721 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b8177-c4c2-454e-8066-4a2da142b176 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:33.887518 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479b8177-c4c2-454e-8066-4a2da142b176 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:33.887829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479b8177-c4c2-454e-8066-4a2da142b176 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:33.889284 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479b8177-c4c2-454e-8066-4a2da142b176 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:33.902189 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 330/641] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:51:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:33.903046 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b8177-c4c2-454e-8066-4a2da142b176 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:33.920282 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d057c-8c36-4645-8e49-266cbdc0c1c1 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:33.961487 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18792722-297c-4dfd-8437-8ed1f2d566a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:33.962314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18792722-297c-4dfd-8437-8ed1f2d566a5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:33.962596 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18792722-297c-4dfd-8437-8ed1f2d566a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:33.962872 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18792722-297c-4dfd-8437-8ed1f2d566a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:33.973550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18792722-297c-4dfd-8437-8ed1f2d566a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:33.981450 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-18792722-297c-4dfd-8437-8ed1f2d566a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:33.981450 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:33.981450 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:33.981450 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:33.981450 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:33.981450 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:33.981450 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:33.981450 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:33.981450 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:33.981450 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:33.981450 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:33.981450 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:33.981450 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:33.981450 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:33.981450 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:33.981450 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:33.982743 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:33.982743 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:33.982743 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:33.982743 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:33.982743 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:33.982743 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:33.982743 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:33.982743 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:33.982743 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:33.982743 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:33.982743 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:33.982743 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:33.982743 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:33.982743 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:33.982743 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:33.982743 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:33.983850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:33.983850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:33.983850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:33.983850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:33.983850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:33.983850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:33.983850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:33.983850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:33.983850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:33.983850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:33.983850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:33.983850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:33.983850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 312/642] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 15:51:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:33.983850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18792722-297c-4dfd-8437-8ed1f2d566a5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:34.019363 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:34.026498 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b8177-c4c2-454e-8066-4a2da142b176 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:34.065798 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda89399-a363-4f75-a852-24552b774b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:34.066464 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda89399-a363-4f75-a852-24552b774b07 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:34.066747 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda89399-a363-4f75-a852-24552b774b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:34.067024 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda89399-a363-4f75-a852-24552b774b07 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:34.067824 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda89399-a363-4f75-a852-24552b774b07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:34.068137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda89399-a363-4f75-a852-24552b774b07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:34.069701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda89399-a363-4f75-a852-24552b774b07 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:34.081747 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 331/643] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:51:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:34.082607 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda89399-a363-4f75-a852-24552b774b07 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:34.094675 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18792722-297c-4dfd-8437-8ed1f2d566a5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:34.117643 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:34.133184 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf579a-09ba-49f7-b05a-3b4fcb6d2a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:34.133901 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf579a-09ba-49f7-b05a-3b4fcb6d2a7e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:34.134186 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf579a-09ba-49f7-b05a-3b4fcb6d2a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:34.134464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf579a-09ba-49f7-b05a-3b4fcb6d2a7e None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:34.135230 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcf579a-09ba-49f7-b05a-3b4fcb6d2a7e None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:34.135539 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcf579a-09ba-49f7-b05a-3b4fcb6d2a7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:34.136929 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcf579a-09ba-49f7-b05a-3b4fcb6d2a7e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:34.145637 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 313/644] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 15:51:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:34.146442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf579a-09ba-49f7-b05a-3b4fcb6d2a7e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:34.192659 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda89399-a363-4f75-a852-24552b774b07 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:34.226997 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aef138-50d3-4edc-9fe5-b5f69453df81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:34.227492 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aef138-50d3-4edc-9fe5-b5f69453df81 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:34.227775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aef138-50d3-4edc-9fe5-b5f69453df81 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:34.228055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aef138-50d3-4edc-9fe5-b5f69453df81 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:34.228787 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aef138-50d3-4edc-9fe5-b5f69453df81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:34.229088 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aef138-50d3-4edc-9fe5-b5f69453df81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:34.230491 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aef138-50d3-4edc-9fe5-b5f69453df81 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:34.247549 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 332/645] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:34.248388 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aef138-50d3-4edc-9fe5-b5f69453df81 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:34.266416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf579a-09ba-49f7-b05a-3b4fcb6d2a7e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:34.271673 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:34.303887 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd1e5a9-f69a-4a05-a0b6-13d5401daa50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:34.304798 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd1e5a9-f69a-4a05-a0b6-13d5401daa50 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:34.305080 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd1e5a9-f69a-4a05-a0b6-13d5401daa50 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:34.305363 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd1e5a9-f69a-4a05-a0b6-13d5401daa50 None admin] PATH_INFO: `/v3/projects/bb1ac00308714c39a79aae09fbb73021/users/f5bfae2cb30843b1b1a0343efef38638/roles/64d94d0d5a7640818c21aa5a622624ab` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:34.326786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd1e5a9-f69a-4a05-a0b6-13d5401daa50 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb1ac00308714c39a79aae09fbb73021, user_id=f5bfae2cb30843b1b1a0343efef38638, role_id=64d94d0d5a7640818c21aa5a622624ab)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:34.328289 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd1e5a9-f69a-4a05-a0b6-13d5401daa50 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:34.342114 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 314/646] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 15:51:34 2022] PUT /identity/v3/projects/bb1ac00308714c39a79aae09fbb73021/users/f5bfae2cb30843b1b1a0343efef38638/roles/64d94d0d5a7640818c21aa5a622624ab => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 15:51:34.342984 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd1e5a9-f69a-4a05-a0b6-13d5401daa50 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:34.380143 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aef138-50d3-4edc-9fe5-b5f69453df81 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:34.387931 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:34.426180 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd46009-e1d4-4ef5-b90e-6a7722a0f30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:34.426875 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd46009-e1d4-4ef5-b90e-6a7722a0f30d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:34.427187 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd46009-e1d4-4ef5-b90e-6a7722a0f30d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:34.427487 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd46009-e1d4-4ef5-b90e-6a7722a0f30d None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:34.428277 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd46009-e1d4-4ef5-b90e-6a7722a0f30d None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:34.428591 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd46009-e1d4-4ef5-b90e-6a7722a0f30d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:34.430074 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd46009-e1d4-4ef5-b90e-6a7722a0f30d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:34.444171 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 333/647] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 15:51:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:34.444171 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd46009-e1d4-4ef5-b90e-6a7722a0f30d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:34.482274 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd1e5a9-f69a-4a05-a0b6-13d5401daa50 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:34.532798 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9485b-0ba8-4102-ac88-834a89d3b32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:34.533689 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9485b-0ba8-4102-ac88-834a89d3b32d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:34.533995 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9485b-0ba8-4102-ac88-834a89d3b32d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:34.534278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9485b-0ba8-4102-ac88-834a89d3b32d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:34.535137 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da9485b-0ba8-4102-ac88-834a89d3b32d None admin] RBAC: Adding query filter params (role.id=64d94d0d5a7640818c21aa5a622624ab, scope.project.id=bb1ac00308714c39a79aae09fbb73021, user.id=092553a86f8848dba8c7271f5c0b2c46) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:34.535457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da9485b-0ba8-4102-ac88-834a89d3b32d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:34.537565 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da9485b-0ba8-4102-ac88-834a89d3b32d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:34.544084 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 315/648] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:51:34 2022] GET /identity/v3/role_assignments?user.id=092553a86f8848dba8c7271f5c0b2c46&scope.project.id=bb1ac00308714c39a79aae09fbb73021&role.id=64d94d0d5a7640818c21aa5a622624ab => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:34.545021 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9485b-0ba8-4102-ac88-834a89d3b32d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:34.555860 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:34.587718 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd46009-e1d4-4ef5-b90e-6a7722a0f30d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:34.644479 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5800b1-0d6b-4794-be14-eb6cf5743ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:34.645562 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5800b1-0d6b-4794-be14-eb6cf5743ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:34.646040 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5800b1-0d6b-4794-be14-eb6cf5743ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:34.646471 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5800b1-0d6b-4794-be14-eb6cf5743ba1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:34.660131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5800b1-0d6b-4794-be14-eb6cf5743ba1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:34.662578 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5800b1-0d6b-4794-be14-eb6cf5743ba1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:34.674312 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:34.683111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-2b5800b1-0d6b-4794-be14-eb6cf5743ba1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:34.683111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:34.683111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:34.683111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:34.683111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:34.683111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:34.683111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:34.683111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:34.683111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:34.683111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:34.683111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:34.683111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:34.683111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:34.683111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:34.683111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:34.684305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:34.684305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:34.684305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:34.684305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:34.684305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:34.684305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:34.684305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:34.684305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:34.684305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:34.684305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:34.684305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:34.684305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:34.684305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:34.684305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:34.684305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:34.684305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:34.684305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:34.685461 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:34.685461 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:34.685461 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:34.685461 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:34.685771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9485b-0ba8-4102-ac88-834a89d3b32d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:34.687730 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 334/649] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:34.688840 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5800b1-0d6b-4794-be14-eb6cf5743ba1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:34.724672 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831e73a-2762-4532-af79-e306f47e3a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:34.725479 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831e73a-2762-4532-af79-e306f47e3a62 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:34.725793 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831e73a-2762-4532-af79-e306f47e3a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:34.726069 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831e73a-2762-4532-af79-e306f47e3a62 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:34.736927 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c831e73a-2762-4532-af79-e306f47e3a62 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:34.738458 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c831e73a-2762-4532-af79-e306f47e3a62 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:34.754823 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-c831e73a-2762-4532-af79-e306f47e3a62 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 15:51:34.754823 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:34.754823 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:34.754823 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:34.754823 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:34.754823 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:34.754823 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:34.754823 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:34.754823 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:34.754823 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:34.754823 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:34.754823 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:34.754823 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:51:34.754823 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:51:34.754823 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:51:34.754823 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:51:34.755874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:34.755874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:34.755874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:51:34.755874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:34.755874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:51:34.755874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:34.755874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:34.755874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:34.755874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:34.755874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:34.755874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:34.755874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:34.755874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:34.755874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:34.755874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:34.755874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:34.755874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:34.755874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:34.756863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:34.756863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:34.756863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:51:34.756863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:51:34.756863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:51:34.756863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:51:34.756863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:51:34.756863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:51:34.756863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 15:51:34.756863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:34.757423 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 316/650] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:51:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:34.758511 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831e73a-2762-4532-af79-e306f47e3a62 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:34.830046 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5800b1-0d6b-4794-be14-eb6cf5743ba1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:34.882767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:34.895642 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c525ee68-878d-41e4-b413-f4d9b15d0ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:34.896454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525ee68-878d-41e4-b413-f4d9b15d0ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:34.896741 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525ee68-878d-41e4-b413-f4d9b15d0ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:34.897498 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525ee68-878d-41e4-b413-f4d9b15d0ea6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:34.902794 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831e73a-2762-4532-af79-e306f47e3a62 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:34.911734 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c525ee68-878d-41e4-b413-f4d9b15d0ea6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:34.913226 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c525ee68-878d-41e4-b413-f4d9b15d0ea6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:34.937407 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-c525ee68-878d-41e4-b413-f4d9b15d0ea6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:34.937407 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:34.937407 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:34.937407 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:34.937407 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:34.937407 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:34.937407 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:34.937407 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:34.937407 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:34.937407 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:34.937407 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:34.937407 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:34.937407 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:34.937407 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:34.937407 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:34.938397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:34.938397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:34.938397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:34.938397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:34.938397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:34.938397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:34.938397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:34.938397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:34.938397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:34.938397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:34.938397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:34.938397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:34.938397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:34.938397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:34.938397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:34.938397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:34.938397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:34.939318 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:34.939318 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:34.939318 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:34.939318 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:34.940067 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 335/651] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:34.941631 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c525ee68-878d-41e4-b413-f4d9b15d0ea6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:34.956445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d450c-67d8-49f3-a485-c85333aa4ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:34.957191 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d450c-67d8-49f3-a485-c85333aa4ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:34.957400 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d450c-67d8-49f3-a485-c85333aa4ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:34.957713 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d450c-67d8-49f3-a485-c85333aa4ac0 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:34.958456 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d450c-67d8-49f3-a485-c85333aa4ac0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:34.958831 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d450c-67d8-49f3-a485-c85333aa4ac0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:34.960274 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d450c-67d8-49f3-a485-c85333aa4ac0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:34.974805 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 317/652] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:34.975792 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d450c-67d8-49f3-a485-c85333aa4ac0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:34.996401 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:35.065256 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c525ee68-878d-41e4-b413-f4d9b15d0ea6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:35.101606 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d450c-67d8-49f3-a485-c85333aa4ac0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:35.108292 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b5c31-8fb3-41a5-897e-1c92d033dec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:35.108978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b5c31-8fb3-41a5-897e-1c92d033dec9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:35.109302 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b5c31-8fb3-41a5-897e-1c92d033dec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:35.109605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b5c31-8fb3-41a5-897e-1c92d033dec9 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:35.110477 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81b5c31-8fb3-41a5-897e-1c92d033dec9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:35.110794 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81b5c31-8fb3-41a5-897e-1c92d033dec9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:35.112281 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81b5c31-8fb3-41a5-897e-1c92d033dec9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:35.137061 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 336/653] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Jul 21 15:51:34 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:35.138042 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b5c31-8fb3-41a5-897e-1c92d033dec9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:35.141323 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354540bd-c60b-4e4e-a06d-8e6be27fdfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:35.142147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354540bd-c60b-4e4e-a06d-8e6be27fdfd2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:35.142435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354540bd-c60b-4e4e-a06d-8e6be27fdfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:35.142717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354540bd-c60b-4e4e-a06d-8e6be27fdfd2 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:35.143458 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354540bd-c60b-4e4e-a06d-8e6be27fdfd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:35.143770 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354540bd-c60b-4e4e-a06d-8e6be27fdfd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:35.145171 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354540bd-c60b-4e4e-a06d-8e6be27fdfd2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:35.159045 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 318/654] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:35.159918 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354540bd-c60b-4e4e-a06d-8e6be27fdfd2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:35.180491 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:35.254040 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b5c31-8fb3-41a5-897e-1c92d033dec9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:35.261023 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:35.291319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07996e-4443-4456-b98d-55b72b3a181a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:35.292134 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07996e-4443-4456-b98d-55b72b3a181a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:35.292420 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07996e-4443-4456-b98d-55b72b3a181a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:35.292700 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07996e-4443-4456-b98d-55b72b3a181a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:35.296558 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354540bd-c60b-4e4e-a06d-8e6be27fdfd2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:35.303348 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc07996e-4443-4456-b98d-55b72b3a181a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:35.305834 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc07996e-4443-4456-b98d-55b72b3a181a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:35.322945 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-dc07996e-4443-4456-b98d-55b72b3a181a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 15:51:35.322945 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:35.322945 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:35.322945 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:35.322945 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:35.322945 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:35.322945 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:35.322945 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:35.322945 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:35.322945 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:35.322945 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:35.322945 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:35.322945 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:51:35.322945 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:51:35.322945 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:51:35.322945 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:51:35.324168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:35.324168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:35.324168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:51:35.324168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:35.324168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:51:35.324168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:35.324168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:35.324168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:35.324168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:35.324168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:35.324168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:35.324168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:35.324168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:35.324168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:35.324168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:35.324168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:35.324168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:35.324168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:35.325366 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:35.325366 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:35.325366 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:51:35.325366 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:51:35.325366 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:51:35.325366 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:51:35.325366 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:51:35.325366 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:51:35.325366 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 15:51:35.325366 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:35.327597 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 337/655] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:51:35 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:35.328581 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07996e-4443-4456-b98d-55b72b3a181a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:35.339763 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7526db3c-7ecd-41ab-9a46-6128f57043e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:35.340605 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526db3c-7ecd-41ab-9a46-6128f57043e8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:35.341505 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526db3c-7ecd-41ab-9a46-6128f57043e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:35.341922 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526db3c-7ecd-41ab-9a46-6128f57043e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:35.438510 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7526db3c-7ecd-41ab-9a46-6128f57043e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:35.442235 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:35.444251 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7526db3c-7ecd-41ab-9a46-6128f57043e8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:35.447035 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07996e-4443-4456-b98d-55b72b3a181a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:35.503106 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d0188-a46d-4b45-824b-f08accd6010d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:35.503985 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d0188-a46d-4b45-824b-f08accd6010d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:35.504325 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d0188-a46d-4b45-824b-f08accd6010d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:35.504545 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d0188-a46d-4b45-824b-f08accd6010d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:35.516301 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86d0188-a46d-4b45-824b-f08accd6010d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:35.517813 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86d0188-a46d-4b45-824b-f08accd6010d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:35.539525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-a86d0188-a46d-4b45-824b-f08accd6010d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 15:51:35.539525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:35.539525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:35.539525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:35.539525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:35.539525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:35.539525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:35.539525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:35.539525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:35.539525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:35.539525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:35.539525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:35.539525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:51:35.539525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:51:35.539525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:51:35.539525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:51:35.540856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:35.540856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:35.540856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:51:35.540856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:35.540856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:51:35.540856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:35.540856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:35.540856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:35.540856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:35.540856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:35.540856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:35.540856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:35.540856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:35.540856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:35.540856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:35.540856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:35.540856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:35.540856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:35.542085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:35.542085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:35.542085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:51:35.542085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:51:35.542085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:51:35.542085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:51:35.542085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:51:35.542085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:51:35.542085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 15:51:35.542085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:35.545668 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 338/656] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 15:51:35 2022] GET /identity/v3/users/neutron => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:35.547353 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d0188-a46d-4b45-824b-f08accd6010d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:35.555742 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:35.587735 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 319/657] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:51:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 15:51:35.589648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7526db3c-7ecd-41ab-9a46-6128f57043e8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:35.668970 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d0188-a46d-4b45-824b-f08accd6010d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:35.716632 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7526db3c-7ecd-41ab-9a46-6128f57043e8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:35.717689 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a0c40-542e-4e65-882b-e20c43b15dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:35.718426 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a0c40-542e-4e65-882b-e20c43b15dcc None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:35.718780 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a0c40-542e-4e65-882b-e20c43b15dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:35.719055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a0c40-542e-4e65-882b-e20c43b15dcc None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:35.720190 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a0c40-542e-4e65-882b-e20c43b15dcc None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:35.720561 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a0c40-542e-4e65-882b-e20c43b15dcc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:35.723181 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a0c40-542e-4e65-882b-e20c43b15dcc None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:35.748735 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 339/658] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Jul 21 15:51:35 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:35.750045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a0c40-542e-4e65-882b-e20c43b15dcc None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:35.757395 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a23e7b0-0422-44f5-8001-b68ea1dd94da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:35.758419 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a23e7b0-0422-44f5-8001-b68ea1dd94da None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:35.758924 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a23e7b0-0422-44f5-8001-b68ea1dd94da None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:35.759407 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a23e7b0-0422-44f5-8001-b68ea1dd94da None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:35.760476 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a23e7b0-0422-44f5-8001-b68ea1dd94da None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:35.760996 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a23e7b0-0422-44f5-8001-b68ea1dd94da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:35.762693 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a23e7b0-0422-44f5-8001-b68ea1dd94da None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:35.772013 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:35.785226 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 320/659] 10.222.0.52 () {56 vars in 1139 bytes} [Thu Jul 21 15:51:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:35.786508 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a23e7b0-0422-44f5-8001-b68ea1dd94da None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:35.838576 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:35.896108 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a0c40-542e-4e65-882b-e20c43b15dcc None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:35.924007 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a23e7b0-0422-44f5-8001-b68ea1dd94da None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:35.945672 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70807f0c-0b10-4af0-9ec6-c6288323d27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:35.946903 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70807f0c-0b10-4af0-9ec6-c6288323d27a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:35.947400 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70807f0c-0b10-4af0-9ec6-c6288323d27a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:35.947954 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70807f0c-0b10-4af0-9ec6-c6288323d27a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:35.959324 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70807f0c-0b10-4af0-9ec6-c6288323d27a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:35.961159 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70807f0c-0b10-4af0-9ec6-c6288323d27a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:35.970113 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6db4ce-0548-40e8-88a7-18a9b1ff823c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:35.970794 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6db4ce-0548-40e8-88a7-18a9b1ff823c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:35.971630 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6db4ce-0548-40e8-88a7-18a9b1ff823c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:35.971923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6db4ce-0548-40e8-88a7-18a9b1ff823c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:35.984296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-70807f0c-0b10-4af0-9ec6-c6288323d27a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:35.984296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:35.984296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:35.984296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:35.984296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:35.984296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:35.984296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:35.984296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:35.984296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:35.984296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:35.984296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:35.984296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:35.984296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:35.984296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:35.984296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:35.985334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:35.985334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:35.985334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:35.985334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:35.985334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:35.985334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:35.985334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:35.985334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:35.985334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:35.985334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:35.985334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:35.985334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:35.985334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:35.985334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:35.985334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:35.985334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:35.985334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:35.986326 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:35.986326 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:35.986326 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:35.986326 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:35.988765 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 340/660] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:35.990013 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70807f0c-0b10-4af0-9ec6-c6288323d27a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:36.058483 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6db4ce-0548-40e8-88a7-18a9b1ff823c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:36.060420 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6db4ce-0548-40e8-88a7-18a9b1ff823c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:36.076665 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:36.106210 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:36.118565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70807f0c-0b10-4af0-9ec6-c6288323d27a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:36.156529 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f503043-d43f-454a-ae0a-cb730a66bad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:36.157606 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f503043-d43f-454a-ae0a-cb730a66bad6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:36.158100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f503043-d43f-454a-ae0a-cb730a66bad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:36.158582 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f503043-d43f-454a-ae0a-cb730a66bad6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:36.198374 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 321/661] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:51:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 15:51:36.199394 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6db4ce-0548-40e8-88a7-18a9b1ff823c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:36.248000 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f503043-d43f-454a-ae0a-cb730a66bad6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:36.249606 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f503043-d43f-454a-ae0a-cb730a66bad6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:36.320568 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:36.332194 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6db4ce-0548-40e8-88a7-18a9b1ff823c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:36.363911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 341/662] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:51:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 15:51:36.364951 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f503043-d43f-454a-ae0a-cb730a66bad6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:36.371474 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f64c20-349b-4169-b560-5341d9b0a3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:36.372374 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f64c20-349b-4169-b560-5341d9b0a3ff None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:36.372920 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f64c20-349b-4169-b560-5341d9b0a3ff None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:36.373350 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f64c20-349b-4169-b560-5341d9b0a3ff None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:36.374449 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f64c20-349b-4169-b560-5341d9b0a3ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:36.375028 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f64c20-349b-4169-b560-5341d9b0a3ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:36.376732 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f64c20-349b-4169-b560-5341d9b0a3ff None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:36.379004 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:36.392531 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 322/663] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:36.394031 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f64c20-349b-4169-b560-5341d9b0a3ff None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:36.475349 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f503043-d43f-454a-ae0a-cb730a66bad6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:36.513820 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f64c20-349b-4169-b560-5341d9b0a3ff None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:36.517829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b7fa8-b333-403c-8d9f-f2372bcf48d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:36.519176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b7fa8-b333-403c-8d9f-f2372bcf48d5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:36.519751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b7fa8-b333-403c-8d9f-f2372bcf48d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:36.520299 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b7fa8-b333-403c-8d9f-f2372bcf48d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:36.533546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155b7fa8-b333-403c-8d9f-f2372bcf48d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:36.535151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155b7fa8-b333-403c-8d9f-f2372bcf48d5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:36.553879 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4105b9-9d5e-4f91-8354-ba04746710aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:36.554583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-155b7fa8-b333-403c-8d9f-f2372bcf48d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:36.554583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:36.554583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:36.554583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:36.554583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:36.554583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:36.554583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:36.554583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:36.554583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:36.554583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:36.554583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:36.554583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:36.554583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:36.554583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:36.554583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:36.555485 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:36.555485 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:36.555485 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:36.555485 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:36.555485 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:36.555485 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:36.555485 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:36.555485 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:36.555485 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:36.555485 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:36.555485 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:36.555485 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:36.555485 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:36.555485 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:36.555485 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:36.555485 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:36.555485 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:36.556410 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:36.556410 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:36.556410 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:36.556410 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:36.557586 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4105b9-9d5e-4f91-8354-ba04746710aa None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:36.558086 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4105b9-9d5e-4f91-8354-ba04746710aa None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:36.558570 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4105b9-9d5e-4f91-8354-ba04746710aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:36.560774 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 342/664] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:36.561749 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b7fa8-b333-403c-8d9f-f2372bcf48d5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:36.573211 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4105b9-9d5e-4f91-8354-ba04746710aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:36.575123 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4105b9-9d5e-4f91-8354-ba04746710aa None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:36.591809 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:36.595766 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-cd4105b9-9d5e-4f91-8354-ba04746710aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:36.595766 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:36.595766 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:36.595766 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:36.595766 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:36.595766 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:36.595766 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:36.595766 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:36.595766 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:36.595766 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:36.595766 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:36.595766 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:36.595766 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:36.595766 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:36.595766 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:36.596714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:36.596714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:36.596714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:36.596714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:36.596714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:36.596714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:36.596714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:36.596714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:36.596714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:36.596714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:36.596714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:36.596714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:36.596714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:36.596714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:36.596714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:36.596714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:36.596714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:36.603112 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:36.603112 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:36.603112 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:36.603112 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:36.603112 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 323/665] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:36.603112 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4105b9-9d5e-4f91-8354-ba04746710aa None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:36.655239 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:36.696351 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b7fa8-b333-403c-8d9f-f2372bcf48d5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:36.729517 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4105b9-9d5e-4f91-8354-ba04746710aa None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:36.736610 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e6e37-a07a-404d-9bc1-3590fc8e804a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:36.740526 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e6e37-a07a-404d-9bc1-3590fc8e804a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:36.740526 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e6e37-a07a-404d-9bc1-3590fc8e804a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:36.740526 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e6e37-a07a-404d-9bc1-3590fc8e804a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:36.757933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261e6e37-a07a-404d-9bc1-3590fc8e804a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:36.759407 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261e6e37-a07a-404d-9bc1-3590fc8e804a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:36.769304 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78d362-dc25-44cb-b4c3-b95e4d30ae25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:36.770516 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78d362-dc25-44cb-b4c3-b95e4d30ae25 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:36.771019 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78d362-dc25-44cb-b4c3-b95e4d30ae25 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:36.771367 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78d362-dc25-44cb-b4c3-b95e4d30ae25 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:36.772123 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da78d362-dc25-44cb-b4c3-b95e4d30ae25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:36.772440 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da78d362-dc25-44cb-b4c3-b95e4d30ae25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:36.773865 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da78d362-dc25-44cb-b4c3-b95e4d30ae25 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:36.774626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-261e6e37-a07a-404d-9bc1-3590fc8e804a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:36.774626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:36.774626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:36.774626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:36.774626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:36.774626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:36.774626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:36.774626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:36.774626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:36.774626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:36.774626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:36.774626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:36.774626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:51:36.774626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:51:36.774626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:51:36.775489 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:51:36.775489 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:36.775489 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:36.775489 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:36.775489 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:36.775489 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:36.775489 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:36.775489 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:36.775489 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:36.775489 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:36.775489 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:36.775489 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:36.775489 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:36.775489 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:36.775489 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:36.775489 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:36.775489 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:36.776408 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:51:36.776408 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:51:36.776408 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:51:36.776408 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:51:36.776408 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:51:36.776408 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:51:36.776408 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:36.776408 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:36.776854 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 343/666] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:51:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:36.777765 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febba51e-3c66-4a5a-ba1d-6d32b0be64e6 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:36.778054 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febba51e-3c66-4a5a-ba1d-6d32b0be64e6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:36.778332 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febba51e-3c66-4a5a-ba1d-6d32b0be64e6 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:36.779758 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 344/667] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:36.780315 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febba51e-3c66-4a5a-ba1d-6d32b0be64e6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:36.799114 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 324/668] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:36.801049 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78d362-dc25-44cb-b4c3-b95e4d30ae25 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:36.876516 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:36.900220 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febba51e-3c66-4a5a-ba1d-6d32b0be64e6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:36.928991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78d362-dc25-44cb-b4c3-b95e4d30ae25 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:36.935737 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f038846a-a395-41bb-b9f6-4e25a343833d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:36.936414 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038846a-a395-41bb-b9f6-4e25a343833d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:36.936698 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038846a-a395-41bb-b9f6-4e25a343833d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:36.936974 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038846a-a395-41bb-b9f6-4e25a343833d None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:36.937767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f038846a-a395-41bb-b9f6-4e25a343833d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:36.938084 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f038846a-a395-41bb-b9f6-4e25a343833d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:36.939486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f038846a-a395-41bb-b9f6-4e25a343833d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:36.941536 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:36.952551 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 345/669] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:36.957916 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da330a39-479f-4055-98fd-07f48adea3bd None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:36.957916 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da330a39-479f-4055-98fd-07f48adea3bd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:36.957916 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da330a39-479f-4055-98fd-07f48adea3bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:36.971854 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-da330a39-479f-4055-98fd-07f48adea3bd None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:36.973328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-da330a39-479f-4055-98fd-07f48adea3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:36.976639 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3b3bb-344b-46f8-91af-8c6dfa2c1d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:36.977313 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3b3bb-344b-46f8-91af-8c6dfa2c1d8b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:36.977619 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3b3bb-344b-46f8-91af-8c6dfa2c1d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:36.977907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3b3bb-344b-46f8-91af-8c6dfa2c1d8b None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:36.978731 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3b3bb-344b-46f8-91af-8c6dfa2c1d8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:36.979044 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3b3bb-344b-46f8-91af-8c6dfa2c1d8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:36.980487 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3b3bb-344b-46f8-91af-8c6dfa2c1d8b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:36.992505 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 325/670] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:51:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:36.993414 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3b3bb-344b-46f8-91af-8c6dfa2c1d8b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:37.050101 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 346/671] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:51:37.052358 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4e068-42d1-423d-8d44-c2815f5fe35e None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:37.052803 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4e068-42d1-423d-8d44-c2815f5fe35e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:37.053274 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4e068-42d1-423d-8d44-c2815f5fe35e None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:37.055558 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 347/672] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:37.056823 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4e068-42d1-423d-8d44-c2815f5fe35e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:37.112347 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3b3bb-344b-46f8-91af-8c6dfa2c1d8b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:37.141733 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:37.146973 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97a7a7-ca2c-4777-b143-4c38f59985b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:37.147778 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97a7a7-ca2c-4777-b143-4c38f59985b2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:37.148063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97a7a7-ca2c-4777-b143-4c38f59985b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:37.148332 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97a7a7-ca2c-4777-b143-4c38f59985b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:37.155465 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de97a7a7-ca2c-4777-b143-4c38f59985b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:37.157480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de97a7a7-ca2c-4777-b143-4c38f59985b2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:37.172443 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-de97a7a7-ca2c-4777-b143-4c38f59985b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:37.172443 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:37.172443 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:37.172443 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:37.172443 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:37.172443 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:37.172443 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:37.172443 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:37.172443 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:37.172443 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:37.172443 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:37.172443 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:37.172443 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:51:37.172443 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:51:37.172443 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:51:37.173874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:51:37.173874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:37.173874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:37.173874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:37.173874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:37.173874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:37.173874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:37.173874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:37.173874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:37.173874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:37.173874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:37.173874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:37.173874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:37.173874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:37.173874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:37.173874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:37.173874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:37.175058 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:51:37.175058 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:51:37.175058 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:51:37.175058 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:51:37.175058 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:51:37.175058 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:51:37.175058 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:37.175058 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:37.175058 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 326/673] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:51:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:37.175058 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97a7a7-ca2c-4777-b143-4c38f59985b2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:37.184237 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4e068-42d1-423d-8d44-c2815f5fe35e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:37.203826 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:37.228143 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22f40dd-a973-4a24-ba9d-77b446087ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:37.228954 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f40dd-a973-4a24-ba9d-77b446087ebe None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:37.229243 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f40dd-a973-4a24-ba9d-77b446087ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:37.229525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f40dd-a973-4a24-ba9d-77b446087ebe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:37.239394 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22f40dd-a973-4a24-ba9d-77b446087ebe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:37.240895 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22f40dd-a973-4a24-ba9d-77b446087ebe None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:37.255831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d22f40dd-a973-4a24-ba9d-77b446087ebe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:37.255831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:37.255831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:37.255831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:37.255831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:37.255831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:37.255831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:37.255831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:37.255831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:37.255831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:37.255831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:37.255831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:37.255831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:51:37.255831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:51:37.255831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:51:37.256885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:51:37.256885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:37.256885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:37.256885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:37.256885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:37.256885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:37.256885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:37.256885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:37.256885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:37.256885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:37.256885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:37.256885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:37.256885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:37.256885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:37.256885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:37.256885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:37.256885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:37.257887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:51:37.257887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:51:37.257887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:51:37.257887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:51:37.257887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:51:37.257887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:51:37.257887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:37.257887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:37.257887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 348/674] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:51:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:37.258393 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b9f0b-387d-425e-b946-2ce476c1d3ff None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:37.258652 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b9f0b-387d-425e-b946-2ce476c1d3ff None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:37.258885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b9f0b-387d-425e-b946-2ce476c1d3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:37.277964 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4d3b9f0b-387d-425e-b946-2ce476c1d3ff None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:37.279289 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4d3b9f0b-387d-425e-b946-2ce476c1d3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:37.286375 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97a7a7-ca2c-4777-b143-4c38f59985b2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:37.321844 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abed354-c294-43cc-a115-2ead19892ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:37.322500 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abed354-c294-43cc-a115-2ead19892ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:37.322786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abed354-c294-43cc-a115-2ead19892ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:37.323065 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abed354-c294-43cc-a115-2ead19892ff4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:37.323910 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abed354-c294-43cc-a115-2ead19892ff4 None admin] RBAC: Adding query filter params (role.id=64d94d0d5a7640818c21aa5a622624ab, scope.project.id=bb1ac00308714c39a79aae09fbb73021, user.id=87c0598af7714aeba08fffbb6805a49e) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:37.324227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abed354-c294-43cc-a115-2ead19892ff4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:37.325702 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abed354-c294-43cc-a115-2ead19892ff4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:37.331914 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 327/675] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:51:37 2022] GET /identity/v3/role_assignments?user.id=87c0598af7714aeba08fffbb6805a49e&scope.project.id=bb1ac00308714c39a79aae09fbb73021&role.id=64d94d0d5a7640818c21aa5a622624ab => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:37.332726 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abed354-c294-43cc-a115-2ead19892ff4 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:37.361276 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 349/676] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:51:37.362505 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b9f0b-387d-425e-b946-2ce476c1d3ff None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:37.459274 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:37.462452 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:37.476133 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abed354-c294-43cc-a115-2ead19892ff4 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:37.505493 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b9f0b-387d-425e-b946-2ce476c1d3ff None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:37.525771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25007f3-3875-4d4f-a70c-5cad903977b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:37.526585 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25007f3-3875-4d4f-a70c-5cad903977b3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:37.526874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25007f3-3875-4d4f-a70c-5cad903977b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:37.527144 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25007f3-3875-4d4f-a70c-5cad903977b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:37.544676 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25007f3-3875-4d4f-a70c-5cad903977b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:37.553255 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-e25007f3-3875-4d4f-a70c-5cad903977b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:37.553255 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:37.553255 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:37.553255 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:37.553255 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:37.553255 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:37.553255 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:37.553255 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:37.553255 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:37.553255 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:37.553255 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:37.553255 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:37.553255 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:37.553255 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:37.553255 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:37.553255 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:37.554339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:37.554339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:37.554339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:37.554339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:37.554339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:37.554339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:37.554339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:37.554339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:37.554339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:37.554339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:37.554339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:37.554339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:37.554339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:37.554339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:37.554339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:37.554339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:37.555227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:37.555227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:37.555227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:37.555227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:37.555227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:37.555227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:37.555227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:37.555227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:37.555227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:37.555227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:37.555227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:37.555227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:37.555905 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 328/677] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 15:51:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:37.556454 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25007f3-3875-4d4f-a70c-5cad903977b3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:37.561853 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918e751-cd18-4e01-8c17-7c1573025fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:37.562688 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918e751-cd18-4e01-8c17-7c1573025fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:37.562969 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918e751-cd18-4e01-8c17-7c1573025fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:37.563239 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918e751-cd18-4e01-8c17-7c1573025fb5 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:37.564038 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0918e751-cd18-4e01-8c17-7c1573025fb5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:37.564376 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0918e751-cd18-4e01-8c17-7c1573025fb5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:37.565872 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0918e751-cd18-4e01-8c17-7c1573025fb5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:37.578661 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 350/678] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:51:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:37.579349 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918e751-cd18-4e01-8c17-7c1573025fb5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:37.680141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25007f3-3875-4d4f-a70c-5cad903977b3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:37.715947 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad73d579-a6b9-4c51-b5ac-9888f3bfa1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:37.716562 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad73d579-a6b9-4c51-b5ac-9888f3bfa1ac None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:37.716822 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad73d579-a6b9-4c51-b5ac-9888f3bfa1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:37.717099 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad73d579-a6b9-4c51-b5ac-9888f3bfa1ac None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:37.717937 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad73d579-a6b9-4c51-b5ac-9888f3bfa1ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:37.718250 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad73d579-a6b9-4c51-b5ac-9888f3bfa1ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:37.720002 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918e751-cd18-4e01-8c17-7c1573025fb5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:37.720461 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad73d579-a6b9-4c51-b5ac-9888f3bfa1ac None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:37.738158 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 329/679] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:51:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:37.738158 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad73d579-a6b9-4c51-b5ac-9888f3bfa1ac None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:37.748768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:37.754803 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:37.772024 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a57a7-4a30-47af-9f73-ce6c786fcd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:37.772839 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a57a7-4a30-47af-9f73-ce6c786fcd5b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:37.773126 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a57a7-4a30-47af-9f73-ce6c786fcd5b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:37.773539 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a57a7-4a30-47af-9f73-ce6c786fcd5b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:37.786215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41a57a7-4a30-47af-9f73-ce6c786fcd5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:37.794638 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-c41a57a7-4a30-47af-9f73-ce6c786fcd5b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:51:37.794638 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:37.794638 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:37.794638 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:37.794638 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:37.794638 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:37.794638 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:37.794638 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:37.794638 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:37.794638 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:37.794638 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:37.794638 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:37.794638 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:37.794638 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:37.794638 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:37.794638 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:37.795665 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:37.795665 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:37.795665 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:37.795665 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:37.795665 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:37.795665 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:37.795665 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:37.795665 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:37.795665 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:37.795665 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:37.795665 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:37.795665 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:37.795665 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:37.795665 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:37.795665 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:37.795665 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:37.796546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:37.796546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:37.796546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:37.796546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:37.796546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:37.796546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:37.796546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:37.796546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:37.796546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:37.796546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:37.796546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:51:37.796546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:37.796546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 351/680] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:51:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:37.797272 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a57a7-4a30-47af-9f73-ce6c786fcd5b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:37.852795 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad73d579-a6b9-4c51-b5ac-9888f3bfa1ac None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:37.891645 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43f3c7-6b06-482e-b4e7-9e099b46aeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:37.892322 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43f3c7-6b06-482e-b4e7-9e099b46aeaa None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:37.893046 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43f3c7-6b06-482e-b4e7-9e099b46aeaa None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:37.893328 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43f3c7-6b06-482e-b4e7-9e099b46aeaa None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:37.898770 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a43f3c7-6b06-482e-b4e7-9e099b46aeaa None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:37.898770 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a43f3c7-6b06-482e-b4e7-9e099b46aeaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:37.898770 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a43f3c7-6b06-482e-b4e7-9e099b46aeaa None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:37.917305 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 330/681] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 15:51:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:37.917305 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43f3c7-6b06-482e-b4e7-9e099b46aeaa None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:37.924446 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a57a7-4a30-47af-9f73-ce6c786fcd5b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:37.983330 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d898c-afc6-475b-b200-9a2576dd5a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:37.984275 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d898c-afc6-475b-b200-9a2576dd5a37 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:37.984724 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d898c-afc6-475b-b200-9a2576dd5a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:37.985164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d898c-afc6-475b-b200-9a2576dd5a37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:37.986764 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814d898c-afc6-475b-b200-9a2576dd5a37 None admin] RBAC: Adding query filter params (role.id=64d94d0d5a7640818c21aa5a622624ab, scope.project.id=bb1ac00308714c39a79aae09fbb73021, user.id=d73a7d5f37e144768d7cc53f13cd5157) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:37.987430 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814d898c-afc6-475b-b200-9a2576dd5a37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:37.989997 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814d898c-afc6-475b-b200-9a2576dd5a37 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:37.991456 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:38.013606 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 352/682] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:51:37 2022] GET /identity/v3/role_assignments?user.id=d73a7d5f37e144768d7cc53f13cd5157&scope.project.id=bb1ac00308714c39a79aae09fbb73021&role.id=64d94d0d5a7640818c21aa5a622624ab => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:38.013606 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d898c-afc6-475b-b200-9a2576dd5a37 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:38.047465 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43f3c7-6b06-482e-b4e7-9e099b46aeaa None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:38.089014 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:38.101458 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58feab-1de1-40a3-85de-42fdd996896d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:38.103174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58feab-1de1-40a3-85de-42fdd996896d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:38.103842 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58feab-1de1-40a3-85de-42fdd996896d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:38.104643 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58feab-1de1-40a3-85de-42fdd996896d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:38.105984 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58feab-1de1-40a3-85de-42fdd996896d None admin] RBAC: Adding query filter params (role.id=64d94d0d5a7640818c21aa5a622624ab, scope.project.id=bb1ac00308714c39a79aae09fbb73021, user.id=1654fdf5ba6b46ba8d9b86a83a76a4b5) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:38.106570 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58feab-1de1-40a3-85de-42fdd996896d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:38.109068 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58feab-1de1-40a3-85de-42fdd996896d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:38.126032 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 331/683] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:51:37 2022] GET /identity/v3/role_assignments?user.id=1654fdf5ba6b46ba8d9b86a83a76a4b5&scope.project.id=bb1ac00308714c39a79aae09fbb73021&role.id=64d94d0d5a7640818c21aa5a622624ab => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:38.127052 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58feab-1de1-40a3-85de-42fdd996896d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:38.184292 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d898c-afc6-475b-b200-9a2576dd5a37 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:38.241449 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e25ebbe-a80c-4e06-85fe-edfab56c04ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:38.242551 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25ebbe-a80c-4e06-85fe-edfab56c04ab None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:38.243052 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25ebbe-a80c-4e06-85fe-edfab56c04ab None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:38.243565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25ebbe-a80c-4e06-85fe-edfab56c04ab None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:38.245253 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e25ebbe-a80c-4e06-85fe-edfab56c04ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:38.245799 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e25ebbe-a80c-4e06-85fe-edfab56c04ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:38.247463 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e25ebbe-a80c-4e06-85fe-edfab56c04ab None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:38.258457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 353/684] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:51:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:38.265329 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e25ebbe-a80c-4e06-85fe-edfab56c04ab None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:38.274983 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58feab-1de1-40a3-85de-42fdd996896d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:38.318703 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a9d74-0b68-421b-8833-ca90eb2caf6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:38.319959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a9d74-0b68-421b-8833-ca90eb2caf6f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:38.320478 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a9d74-0b68-421b-8833-ca90eb2caf6f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:38.321026 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a9d74-0b68-421b-8833-ca90eb2caf6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:38.324717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:38.336503 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a9d74-0b68-421b-8833-ca90eb2caf6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:38.338382 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a9d74-0b68-421b-8833-ca90eb2caf6f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:38.359969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-d85a9d74-0b68-421b-8833-ca90eb2caf6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:38.359969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:38.359969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:38.359969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:38.359969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:38.359969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:38.359969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:38.359969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:38.359969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:38.359969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:38.359969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:38.359969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:38.359969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:38.359969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:38.359969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:38.360966 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:38.360966 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:38.360966 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:38.360966 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:38.360966 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:38.360966 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:38.360966 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:38.360966 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:38.360966 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:38.360966 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:38.360966 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:38.360966 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:38.360966 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:38.360966 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:38.360966 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:38.360966 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:38.360966 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:38.361912 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:38.361912 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:38.361912 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:38.361912 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:38.364241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 332/685] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:38.369472 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a9d74-0b68-421b-8833-ca90eb2caf6f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:38.398328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e25ebbe-a80c-4e06-85fe-edfab56c04ab None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:38.406410 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:38.437362 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea489d-1072-4815-ba1a-45285067afdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:38.438227 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea489d-1072-4815-ba1a-45285067afdf None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:38.438499 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea489d-1072-4815-ba1a-45285067afdf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:38.438775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea489d-1072-4815-ba1a-45285067afdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:38.463618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ea489d-1072-4815-ba1a-45285067afdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:38.465243 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ea489d-1072-4815-ba1a-45285067afdf None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:38.485618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e0ea489d-1072-4815-ba1a-45285067afdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:38.485618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:38.485618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:38.485618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:38.485618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:38.485618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:38.485618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:38.485618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:38.485618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:38.485618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:38.485618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:38.485618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:38.485618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:38.485618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:38.485618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:38.486598 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:38.486598 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:38.486598 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:38.486598 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:38.486598 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:38.486598 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:38.486598 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:38.486598 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:38.486598 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:38.486598 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:38.486598 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:38.486598 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:38.486598 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:38.486598 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:38.486598 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:38.486598 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:38.486598 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:38.487533 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:38.487533 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:38.487533 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:38.487533 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:38.488291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 354/686] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:38.493703 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea489d-1072-4815-ba1a-45285067afdf None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:38.513545 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a9d74-0b68-421b-8833-ca90eb2caf6f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:38.555558 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d1c48-12e5-46b9-b693-f43cf435276b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:38.556386 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d1c48-12e5-46b9-b693-f43cf435276b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:38.557032 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d1c48-12e5-46b9-b693-f43cf435276b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:38.557320 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d1c48-12e5-46b9-b693-f43cf435276b None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:38.558444 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d1c48-12e5-46b9-b693-f43cf435276b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:38.558763 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d1c48-12e5-46b9-b693-f43cf435276b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:38.560769 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d1c48-12e5-46b9-b693-f43cf435276b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:38.581667 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 333/687] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:38.584930 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d1c48-12e5-46b9-b693-f43cf435276b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:38.632867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea489d-1072-4815-ba1a-45285067afdf None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:38.654075 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:38.678675 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33111a-c276-4c28-8f64-b359ee06a338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:38.679937 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33111a-c276-4c28-8f64-b359ee06a338 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:38.681416 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33111a-c276-4c28-8f64-b359ee06a338 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:38.681954 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33111a-c276-4c28-8f64-b359ee06a338 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:38.682892 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f33111a-c276-4c28-8f64-b359ee06a338 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:38.683389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f33111a-c276-4c28-8f64-b359ee06a338 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:38.685195 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f33111a-c276-4c28-8f64-b359ee06a338 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:38.702571 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 355/688] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:38.708301 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33111a-c276-4c28-8f64-b359ee06a338 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:38.714661 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d1c48-12e5-46b9-b693-f43cf435276b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:38.731650 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:38.769733 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515e1aa-ccdd-4eee-88c7-5784468788bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:38.771129 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515e1aa-ccdd-4eee-88c7-5784468788bd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:38.771579 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515e1aa-ccdd-4eee-88c7-5784468788bd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:38.772000 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515e1aa-ccdd-4eee-88c7-5784468788bd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:38.782693 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9515e1aa-ccdd-4eee-88c7-5784468788bd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:38.784448 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9515e1aa-ccdd-4eee-88c7-5784468788bd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:38.816157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-9515e1aa-ccdd-4eee-88c7-5784468788bd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 15:51:38.816157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:38.816157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:38.816157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:38.816157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:38.816157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:38.816157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:38.816157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:38.816157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:38.816157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:38.816157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:38.816157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:38.816157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:51:38.816157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:51:38.816157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:51:38.816157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:51:38.817195 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:38.817195 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:38.817195 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:51:38.817195 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:38.817195 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:51:38.817195 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:38.817195 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:38.817195 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:38.817195 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:38.817195 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:38.817195 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:38.817195 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:38.817195 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:38.817195 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:38.817195 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:38.817195 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:38.817195 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:38.817195 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:38.818190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:38.818190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:38.818190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:51:38.818190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:51:38.818190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:51:38.818190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:51:38.818190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:51:38.818190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:51:38.818190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 15:51:38.818190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:38.818190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 334/689] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:51:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:38.830631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515e1aa-ccdd-4eee-88c7-5784468788bd None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:38.901389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33111a-c276-4c28-8f64-b359ee06a338 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:38.981716 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ea4a2-542d-4c7b-906e-98598776e4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:38.982606 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ea4a2-542d-4c7b-906e-98598776e4dc None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:38.982890 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ea4a2-542d-4c7b-906e-98598776e4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:38.983215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ea4a2-542d-4c7b-906e-98598776e4dc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:38.997311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ea4a2-542d-4c7b-906e-98598776e4dc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:39.001050 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ea4a2-542d-4c7b-906e-98598776e4dc None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:39.028115 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-603ea4a2-542d-4c7b-906e-98598776e4dc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 15:51:39.028115 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:39.028115 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:39.028115 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:39.028115 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:39.028115 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:39.028115 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:39.028115 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:39.028115 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:39.028115 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:39.028115 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:39.028115 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:39.028115 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:51:39.028115 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:51:39.028115 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:51:39.028115 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:51:39.029100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:39.029100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:39.029100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:51:39.029100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:39.029100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:51:39.029100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:39.029100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:39.029100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:39.029100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:39.029100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:39.029100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:39.029100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:39.029100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:39.029100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:39.029100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:39.029100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:39.029100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:39.029100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:39.030151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:39.030151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:39.030151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:51:39.030151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:51:39.030151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:51:39.030151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:51:39.030151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:51:39.030151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:51:39.030151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 15:51:39.030151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:39.030695 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 356/690] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jul 21 15:51:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:51:39.037114 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ea4a2-542d-4c7b-906e-98598776e4dc None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:39.048501 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515e1aa-ccdd-4eee-88c7-5784468788bd None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:39.072269 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:39.140997 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:39.167630 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ea4a2-542d-4c7b-906e-98598776e4dc None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:39.206676 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f06b5-38f9-4161-9dc9-5ea27fc8ef4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:39.207695 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f06b5-38f9-4161-9dc9-5ea27fc8ef4c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:39.208188 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f06b5-38f9-4161-9dc9-5ea27fc8ef4c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:39.208618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f06b5-38f9-4161-9dc9-5ea27fc8ef4c None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:39.209012 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3faaabf-8adc-4a45-9829-5a17919e937f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:39.209837 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3faaabf-8adc-4a45-9829-5a17919e937f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:39.210022 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f06b5-38f9-4161-9dc9-5ea27fc8ef4c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:39.210301 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3faaabf-8adc-4a45-9829-5a17919e937f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:39.210511 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f06b5-38f9-4161-9dc9-5ea27fc8ef4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:39.210752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3faaabf-8adc-4a45-9829-5a17919e937f None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:39.212332 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3faaabf-8adc-4a45-9829-5a17919e937f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:39.212455 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f06b5-38f9-4161-9dc9-5ea27fc8ef4c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:39.213273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3faaabf-8adc-4a45-9829-5a17919e937f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:39.215163 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3faaabf-8adc-4a45-9829-5a17919e937f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:39.240147 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 357/691] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jul 21 15:51:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:39.251232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3faaabf-8adc-4a45-9829-5a17919e937f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:39.269351 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 335/692] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Jul 21 15:51:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 443 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:39.289210 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f06b5-38f9-4161-9dc9-5ea27fc8ef4c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:39.458261 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f06b5-38f9-4161-9dc9-5ea27fc8ef4c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:39.502051 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ecc52-18cb-4ed1-ac98-7e0aba0a5f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:39.502749 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ecc52-18cb-4ed1-ac98-7e0aba0a5f7c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:39.503546 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ecc52-18cb-4ed1-ac98-7e0aba0a5f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:39.503833 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ecc52-18cb-4ed1-ac98-7e0aba0a5f7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:39.516495 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3faaabf-8adc-4a45-9829-5a17919e937f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:39.584500 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8ecc52-18cb-4ed1-ac98-7e0aba0a5f7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:39.586803 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8ecc52-18cb-4ed1-ac98-7e0aba0a5f7c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:39.596979 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:39.666528 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60826b67-981d-42de-9a11-3dcb77c04183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:39.667217 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60826b67-981d-42de-9a11-3dcb77c04183 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:39.667512 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60826b67-981d-42de-9a11-3dcb77c04183 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:39.667795 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60826b67-981d-42de-9a11-3dcb77c04183 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:39.683877 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:39.721119 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 336/693] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:51:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 15:51:39.727156 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ecc52-18cb-4ed1-ac98-7e0aba0a5f7c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:39.804089 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:39.849756 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60826b67-981d-42de-9a11-3dcb77c04183 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:39.852101 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60826b67-981d-42de-9a11-3dcb77c04183 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:39.854240 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ecc52-18cb-4ed1-ac98-7e0aba0a5f7c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:39.906851 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f9200-41b2-481c-8552-7166563392b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:39.907780 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f9200-41b2-481c-8552-7166563392b0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:39.908130 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f9200-41b2-481c-8552-7166563392b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:39.908471 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f9200-41b2-481c-8552-7166563392b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:39.924871 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56f9200-41b2-481c-8552-7166563392b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:39.926372 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56f9200-41b2-481c-8552-7166563392b0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:39.949141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-f56f9200-41b2-481c-8552-7166563392b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:39.949141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:39.949141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:39.949141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:39.949141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:39.949141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:39.949141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:39.949141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:39.949141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:39.949141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:39.949141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:39.949141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:39.949141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:39.949141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:39.949141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:39.950123 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:39.950123 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:39.950123 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:39.950123 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:39.950123 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:39.950123 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:39.950123 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:39.950123 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:39.950123 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:39.950123 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:39.950123 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:39.950123 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:39.950123 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:39.950123 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:39.950123 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:39.950123 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:39.950123 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:39.951033 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:39.951033 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:39.951033 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:39.951033 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:39.953673 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 337/694] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:51:39.955045 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8516556-b09e-4beb-a7ad-9636f1ef9396 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:39.955445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8516556-b09e-4beb-a7ad-9636f1ef9396 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:39.955920 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8516556-b09e-4beb-a7ad-9636f1ef9396 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:39.957554 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 338/695] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:39.958291 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8516556-b09e-4beb-a7ad-9636f1ef9396 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:40.007410 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 358/696] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:51:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 757 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 15:51:40.008852 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68792af5-b03f-4644-a1fd-1e86c76fb50f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:40.009148 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68792af5-b03f-4644-a1fd-1e86c76fb50f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:40.009432 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68792af5-b03f-4644-a1fd-1e86c76fb50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:40.028327 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-68792af5-b03f-4644-a1fd-1e86c76fb50f None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:40.029639 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-68792af5-b03f-4644-a1fd-1e86c76fb50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:40.064647 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:40.116959 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 359/697] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:51:40.118045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68792af5-b03f-4644-a1fd-1e86c76fb50f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:40.119165 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8516556-b09e-4beb-a7ad-9636f1ef9396 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:40.152100 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:40.180840 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a218538-9911-48b3-8ea5-1ebbb45f9894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:40.182350 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a218538-9911-48b3-8ea5-1ebbb45f9894 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:40.182825 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a218538-9911-48b3-8ea5-1ebbb45f9894 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:40.183496 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a218538-9911-48b3-8ea5-1ebbb45f9894 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:40.185742 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a218538-9911-48b3-8ea5-1ebbb45f9894 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:40.186306 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a218538-9911-48b3-8ea5-1ebbb45f9894 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:40.189234 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a218538-9911-48b3-8ea5-1ebbb45f9894 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:40.212348 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 339/698] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:40.213256 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a218538-9911-48b3-8ea5-1ebbb45f9894 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:40.259197 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68792af5-b03f-4644-a1fd-1e86c76fb50f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:40.305208 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b0826-9f83-4e23-af92-34c87d58869c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:40.306245 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b0826-9f83-4e23-af92-34c87d58869c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:40.306535 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b0826-9f83-4e23-af92-34c87d58869c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:40.306813 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b0826-9f83-4e23-af92-34c87d58869c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:40.321316 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0b0826-9f83-4e23-af92-34c87d58869c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:40.323349 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0b0826-9f83-4e23-af92-34c87d58869c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:40.348176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-2e0b0826-9f83-4e23-af92-34c87d58869c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:40.348176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:40.348176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:40.348176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:40.348176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:40.348176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:40.348176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:40.348176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:40.348176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:40.348176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:40.348176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:40.348176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:40.348176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:40.348176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:40.348176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:40.349192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:40.349192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:40.349192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:40.349192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:40.349192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:40.349192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:40.349192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:40.349192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:40.349192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:40.349192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:40.349192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:40.349192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:40.349192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:40.349192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:40.349192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:40.349192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:40.349192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:40.350374 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:40.350374 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:40.350374 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:40.350374 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:40.350374 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 360/699] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:40.351052 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a218538-9911-48b3-8ea5-1ebbb45f9894 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:40.351133 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b0826-9f83-4e23-af92-34c87d58869c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:40.380997 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:40.396989 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a1f678-7a46-4fd2-babf-6bee101c15c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:40.397845 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1f678-7a46-4fd2-babf-6bee101c15c6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:40.398131 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1f678-7a46-4fd2-babf-6bee101c15c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:40.398410 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1f678-7a46-4fd2-babf-6bee101c15c6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:40.399181 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a1f678-7a46-4fd2-babf-6bee101c15c6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:40.400686 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a1f678-7a46-4fd2-babf-6bee101c15c6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:40.417532 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b3a1f678-7a46-4fd2-babf-6bee101c15c6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 15:51:40.417532 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:40.417532 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 15:51:40.417532 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 15:51:40.417532 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 15:51:40.417532 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 15:51:40.417532 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 15:51:40.417532 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:51:40.417532 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 15:51:40.417532 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:40.417532 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 15:51:40.417532 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:40.417532 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:40.417532 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:40.417532 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:40.417532 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:40.418836 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:40.418836 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:40.418836 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:40.418836 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:40.418836 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:40.418836 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:40.418836 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:40.418836 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 15:51:40.418836 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 15:51:40.418836 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 15:51:40.418836 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 15:51:40.418836 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:40.418836 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:40.418836 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:40.418836 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:40.418836 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:40.419878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:40.419878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:40.419878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:40.419878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:40.419878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:40.419878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:40.419878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:40.419878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:40.419878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:40.419878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:40.419878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:40.419878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 15:51:40.419878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:51:40.419878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 15:51:40.419878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:40.419878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 340/700] 10.222.0.52 () {56 vars in 1129 bytes} [Thu Jul 21 15:51:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:40.420768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a1f678-7a46-4fd2-babf-6bee101c15c6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:40.477755 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:40.507115 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b0826-9f83-4e23-af92-34c87d58869c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:40.553432 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a1f678-7a46-4fd2-babf-6bee101c15c6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:40.559619 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495bd09d-fbbd-4995-b759-ce3093c97a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:40.560531 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495bd09d-fbbd-4995-b759-ce3093c97a6c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:40.560812 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495bd09d-fbbd-4995-b759-ce3093c97a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:40.561082 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495bd09d-fbbd-4995-b759-ce3093c97a6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:40.570206 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495bd09d-fbbd-4995-b759-ce3093c97a6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:40.571831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495bd09d-fbbd-4995-b759-ce3093c97a6c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:40.587342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-495bd09d-fbbd-4995-b759-ce3093c97a6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:40.587342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:40.587342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:40.587342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:40.587342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:40.587342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:40.587342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:40.587342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:40.587342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:40.587342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:40.587342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:40.587342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:40.587342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:51:40.587342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:51:40.587342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:51:40.588371 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:51:40.588371 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:40.588371 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:40.588371 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:40.588371 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:40.588371 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:40.588371 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:40.588371 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:40.588371 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:40.588371 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:40.588371 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:40.588371 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:40.588371 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:40.588371 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:40.588371 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:40.588371 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:40.588371 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:40.589317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:51:40.589317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:51:40.589317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:51:40.589317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:51:40.589317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:51:40.589317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:51:40.589317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:40.589317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:40.594018 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 361/701] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:51:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:40.594976 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495bd09d-fbbd-4995-b759-ce3093c97a6c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:40.602637 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e0f526-8260-4ea8-8ea4-b02c4fefa02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:40.603699 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0f526-8260-4ea8-8ea4-b02c4fefa02e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:40.604278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0f526-8260-4ea8-8ea4-b02c4fefa02e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:40.604755 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0f526-8260-4ea8-8ea4-b02c4fefa02e None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:40.606384 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e0f526-8260-4ea8-8ea4-b02c4fefa02e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:40.608261 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e0f526-8260-4ea8-8ea4-b02c4fefa02e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:40.610091 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e0f526-8260-4ea8-8ea4-b02c4fefa02e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:40.630615 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 341/702] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:40.631529 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e0f526-8260-4ea8-8ea4-b02c4fefa02e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:40.698096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:40.721752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495bd09d-fbbd-4995-b759-ce3093c97a6c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:40.754587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e0f526-8260-4ea8-8ea4-b02c4fefa02e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:40.756151 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:40.757285 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a36c0-1b5e-45ab-8cef-4057ff9fb323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:40.758123 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a36c0-1b5e-45ab-8cef-4057ff9fb323 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:40.758576 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a36c0-1b5e-45ab-8cef-4057ff9fb323 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:40.758853 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a36c0-1b5e-45ab-8cef-4057ff9fb323 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:40.759563 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a36c0-1b5e-45ab-8cef-4057ff9fb323 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:40.759943 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a36c0-1b5e-45ab-8cef-4057ff9fb323 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:40.761250 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a36c0-1b5e-45ab-8cef-4057ff9fb323 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:40.766031 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 362/703] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Jul 21 15:51:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:40.766841 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a36c0-1b5e-45ab-8cef-4057ff9fb323 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:40.792411 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5835943-058f-495f-aa6b-ccc61c7dda4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:40.792853 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5835943-058f-495f-aa6b-ccc61c7dda4d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:40.793145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5835943-058f-495f-aa6b-ccc61c7dda4d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:40.793458 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5835943-058f-495f-aa6b-ccc61c7dda4d None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:40.794298 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5835943-058f-495f-aa6b-ccc61c7dda4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:40.794616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5835943-058f-495f-aa6b-ccc61c7dda4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:40.796061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5835943-058f-495f-aa6b-ccc61c7dda4d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:40.808872 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 342/704] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:51:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:40.810290 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b554e30-bc73-49ae-991b-625c5a736058 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:40.810645 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b554e30-bc73-49ae-991b-625c5a736058 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:40.811095 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b554e30-bc73-49ae-991b-625c5a736058 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:40.813453 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 343/705] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:40.816511 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e83aea-89df-4854-95af-cd2103084381 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:40.816949 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e83aea-89df-4854-95af-cd2103084381 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:40.817380 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e83aea-89df-4854-95af-cd2103084381 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:40.821085 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 344/706] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:40.821085 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e83aea-89df-4854-95af-cd2103084381 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:40.872186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a36c0-1b5e-45ab-8cef-4057ff9fb323 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:40.909799 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db32011-bef4-47f5-90a2-cdf3272ee760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:40.910670 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db32011-bef4-47f5-90a2-cdf3272ee760 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:40.910954 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db32011-bef4-47f5-90a2-cdf3272ee760 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:40.911232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db32011-bef4-47f5-90a2-cdf3272ee760 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:40.918622 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db32011-bef4-47f5-90a2-cdf3272ee760 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:40.920063 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db32011-bef4-47f5-90a2-cdf3272ee760 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:40.935450 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-3db32011-bef4-47f5-90a2-cdf3272ee760 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:40.935450 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:40.935450 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:40.935450 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:40.935450 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:40.935450 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:40.935450 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:40.935450 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:40.935450 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:40.935450 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:40.935450 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:40.935450 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:40.935450 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:51:40.935450 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:51:40.935450 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:51:40.936478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:51:40.936478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:40.936478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:40.936478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:40.936478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:40.936478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:40.936478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:40.936478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:40.936478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:40.936478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:40.936478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:40.936478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:40.936478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:40.936478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:40.936478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:40.936478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:40.936478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:40.943300 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:51:40.943300 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:51:40.943300 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:51:40.943300 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:51:40.943300 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:51:40.943300 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:51:40.943300 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:40.943300 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:40.943300 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 363/707] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:51:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:40.943300 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db32011-bef4-47f5-90a2-cdf3272ee760 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:40.943300 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:40.948251 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e83aea-89df-4854-95af-cd2103084381 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:40.986991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30680bc-3104-43cb-b92e-e6996ff0346b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:40.987668 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30680bc-3104-43cb-b92e-e6996ff0346b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:40.987959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30680bc-3104-43cb-b92e-e6996ff0346b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:40.988241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30680bc-3104-43cb-b92e-e6996ff0346b None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:40.988970 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30680bc-3104-43cb-b92e-e6996ff0346b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:40.989281 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30680bc-3104-43cb-b92e-e6996ff0346b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:40.990728 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30680bc-3104-43cb-b92e-e6996ff0346b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:40.996738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 345/708] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Jul 21 15:51:40 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:40.998142 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4811b-b1d5-4d39-97a8-d7e829ab4be9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:40.998446 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4811b-b1d5-4d39-97a8-d7e829ab4be9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:40.998734 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4811b-b1d5-4d39-97a8-d7e829ab4be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:41.017152 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-dac4811b-b1d5-4d39-97a8-d7e829ab4be9 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:41.018531 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-dac4811b-b1d5-4d39-97a8-d7e829ab4be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:41.053531 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:41.078599 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db32011-bef4-47f5-90a2-cdf3272ee760 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:41.123504 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 346/709] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:51:41.129135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb3359-a94d-4e96-8172-020c1fb4d3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:41.129954 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb3359-a94d-4e96-8172-020c1fb4d3a8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:41.130326 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb3359-a94d-4e96-8172-020c1fb4d3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:41.130675 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb3359-a94d-4e96-8172-020c1fb4d3a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:41.131599 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbb3359-a94d-4e96-8172-020c1fb4d3a8 None admin] RBAC: Adding query filter params (role.id=64d94d0d5a7640818c21aa5a622624ab, scope.project.id=bb1ac00308714c39a79aae09fbb73021, user.id=f5bfae2cb30843b1b1a0343efef38638) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:41.131950 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4b34a-dc8f-4e8d-b97c-d6c758a091ee None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:41.132057 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbb3359-a94d-4e96-8172-020c1fb4d3a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:41.132551 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4b34a-dc8f-4e8d-b97c-d6c758a091ee None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:41.133029 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4b34a-dc8f-4e8d-b97c-d6c758a091ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:41.133879 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbb3359-a94d-4e96-8172-020c1fb4d3a8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:41.146077 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 364/710] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:51:40 2022] GET /identity/v3/role_assignments?user.id=f5bfae2cb30843b1b1a0343efef38638&scope.project.id=bb1ac00308714c39a79aae09fbb73021&role.id=64d94d0d5a7640818c21aa5a622624ab => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:41.147363 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb3359-a94d-4e96-8172-020c1fb4d3a8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:41.156849 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-29d4b34a-dc8f-4e8d-b97c-d6c758a091ee None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:41.158139 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-29d4b34a-dc8f-4e8d-b97c-d6c758a091ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:41.249600 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 347/711] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:51:41.249600 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d4b34a-dc8f-4e8d-b97c-d6c758a091ee None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:41.261617 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:41.318133 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb3359-a94d-4e96-8172-020c1fb4d3a8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:41.361114 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34a23f9-ec79-4a5a-aaa2-036d8ae70705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:41.361830 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a23f9-ec79-4a5a-aaa2-036d8ae70705 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:41.362109 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a23f9-ec79-4a5a-aaa2-036d8ae70705 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:41.362378 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a23f9-ec79-4a5a-aaa2-036d8ae70705 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:41.363193 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34a23f9-ec79-4a5a-aaa2-036d8ae70705 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:41.363503 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34a23f9-ec79-4a5a-aaa2-036d8ae70705 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:41.365005 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34a23f9-ec79-4a5a-aaa2-036d8ae70705 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:41.373624 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:41.383045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 365/712] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:51:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:41.383725 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34a23f9-ec79-4a5a-aaa2-036d8ae70705 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:41.395820 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d4b34a-dc8f-4e8d-b97c-d6c758a091ee None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:41.442374 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f573cd7a-d677-4320-ad32-868a38c4d51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:41.443190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573cd7a-d677-4320-ad32-868a38c4d51b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:41.443470 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573cd7a-d677-4320-ad32-868a38c4d51b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:41.443742 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573cd7a-d677-4320-ad32-868a38c4d51b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:41.456351 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f573cd7a-d677-4320-ad32-868a38c4d51b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:41.458829 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f573cd7a-d677-4320-ad32-868a38c4d51b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:41.484019 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-f573cd7a-d677-4320-ad32-868a38c4d51b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:41.484019 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:41.484019 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:41.484019 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:41.484019 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:41.484019 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:41.484019 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:41.484019 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:41.484019 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:41.484019 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:41.484019 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:41.484019 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:41.484019 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:41.484019 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:41.484019 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:41.485004 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:41.485004 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:41.485004 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:41.485004 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:41.485004 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:41.485004 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:41.485004 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:41.485004 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:41.485004 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:41.485004 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:41.485004 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:41.485004 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:41.485004 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:41.485004 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:41.485004 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:41.485004 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:41.485004 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:41.490750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:41.490750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:41.490750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:41.490750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:41.490750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 348/713] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:41.490750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f573cd7a-d677-4320-ad32-868a38c4d51b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:41.519989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34a23f9-ec79-4a5a-aaa2-036d8ae70705 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:41.552662 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:41.564346 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a6cbd-4030-4f90-b9b7-63ded3b8eb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:41.565349 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a6cbd-4030-4f90-b9b7-63ded3b8eb82 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:41.565984 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a6cbd-4030-4f90-b9b7-63ded3b8eb82 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:41.566262 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a6cbd-4030-4f90-b9b7-63ded3b8eb82 None admin] PATH_INFO: `/v3/services/network` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:41.567187 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a6cbd-4030-4f90-b9b7-63ded3b8eb82 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:41.569020 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a6cbd-4030-4f90-b9b7-63ded3b8eb82 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:41.585920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-7e4a6cbd-4030-4f90-b9b7-63ded3b8eb82 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 15:51:41.585920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:41.585920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 15:51:41.585920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 15:51:41.585920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 15:51:41.585920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 15:51:41.585920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 15:51:41.585920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:51:41.585920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 15:51:41.585920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:41.585920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 15:51:41.585920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:41.585920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:41.585920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:41.585920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:41.585920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:41.585920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:41.587950 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:41.587950 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:41.587950 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:41.587950 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:41.587950 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:41.587950 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:41.587950 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 15:51:41.587950 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 15:51:41.587950 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 15:51:41.587950 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 15:51:41.587950 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:41.587950 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:41.587950 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:41.587950 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:41.587950 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:41.587950 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:41.590177 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:41.590177 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:41.590177 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:41.590177 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:41.590177 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:41.590177 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:41.590177 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:41.590177 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:41.590177 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:41.590177 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:41.590177 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 15:51:41.590177 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:51:41.590177 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 15:51:41.590177 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:41.590177 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 366/714] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:51:41 2022] GET /identity/v3/services/network => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:41.592183 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a6cbd-4030-4f90-b9b7-63ded3b8eb82 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:41.616524 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f573cd7a-d677-4320-ad32-868a38c4d51b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:41.655574 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a280532-2002-4099-87c7-52b17f08ff42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:41.656708 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a280532-2002-4099-87c7-52b17f08ff42 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:41.657229 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a280532-2002-4099-87c7-52b17f08ff42 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:41.657888 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a280532-2002-4099-87c7-52b17f08ff42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:41.659196 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a280532-2002-4099-87c7-52b17f08ff42 None admin] RBAC: Adding query filter params (role.id=475521b8a0174037b162f92022aea1a1, scope.project.id=bb1ac00308714c39a79aae09fbb73021, user.id=092553a86f8848dba8c7271f5c0b2c46) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:41.659763 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a280532-2002-4099-87c7-52b17f08ff42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:41.661443 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a280532-2002-4099-87c7-52b17f08ff42 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:41.668973 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 349/715] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:51:41 2022] GET /identity/v3/role_assignments?user.id=092553a86f8848dba8c7271f5c0b2c46&scope.project.id=bb1ac00308714c39a79aae09fbb73021&role.id=475521b8a0174037b162f92022aea1a1 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:41.674839 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a280532-2002-4099-87c7-52b17f08ff42 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:41.681616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:41.730172 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a6cbd-4030-4f90-b9b7-63ded3b8eb82 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:41.775779 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad3645-6d0e-4370-9466-9d70f7057ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:41.777612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad3645-6d0e-4370-9466-9d70f7057ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:41.777612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad3645-6d0e-4370-9466-9d70f7057ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:41.777612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad3645-6d0e-4370-9466-9d70f7057ee2 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:41.778013 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad3645-6d0e-4370-9466-9d70f7057ee2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:41.778323 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad3645-6d0e-4370-9466-9d70f7057ee2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:41.781263 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad3645-6d0e-4370-9466-9d70f7057ee2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:41.797150 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 367/716] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:41.803660 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad3645-6d0e-4370-9466-9d70f7057ee2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:41.870369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:41.887868 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a280532-2002-4099-87c7-52b17f08ff42 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:41.931206 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad3645-6d0e-4370-9466-9d70f7057ee2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:41.932426 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74113345-fef4-4dd7-9585-e721a5024fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:41.933544 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74113345-fef4-4dd7-9585-e721a5024fdf None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:41.934050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74113345-fef4-4dd7-9585-e721a5024fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:41.935444 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74113345-fef4-4dd7-9585-e721a5024fdf None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:41.936327 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74113345-fef4-4dd7-9585-e721a5024fdf None admin] RBAC: Adding query filter params (name=network) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:41.936801 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74113345-fef4-4dd7-9585-e721a5024fdf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:41.938897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74113345-fef4-4dd7-9585-e721a5024fdf None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:41.945828 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 350/717] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 21 15:51:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:41.955619 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74113345-fef4-4dd7-9585-e721a5024fdf None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:42.010471 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ae744-50ed-4f10-914e-46db1aa4ecec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:42.011151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ae744-50ed-4f10-914e-46db1aa4ecec None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:42.011428 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ae744-50ed-4f10-914e-46db1aa4ecec None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:42.011700 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ae744-50ed-4f10-914e-46db1aa4ecec None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:42.012415 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2ae744-50ed-4f10-914e-46db1aa4ecec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:42.014043 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2ae744-50ed-4f10-914e-46db1aa4ecec None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:42.060776 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 368/718] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 15:51:41 2022] POST /identity/v3/users => generated 254 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:51:42.084293 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:42.133508 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74113345-fef4-4dd7-9585-e721a5024fdf None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:42.207167 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb74c7-c23b-47d1-a562-381e579fa12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:42.208403 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb74c7-c23b-47d1-a562-381e579fa12d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:42.208703 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb74c7-c23b-47d1-a562-381e579fa12d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:42.208982 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb74c7-c23b-47d1-a562-381e579fa12d None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:42.209782 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bb74c7-c23b-47d1-a562-381e579fa12d None admin] RBAC: Adding query filter params (type=network) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:42.210123 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bb74c7-c23b-47d1-a562-381e579fa12d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:42.211571 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bb74c7-c23b-47d1-a562-381e579fa12d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:42.217210 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 351/719] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 21 15:51:41 2022] GET /identity/v3/services?type=network => generated 124 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:43.545823 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25769f87-1e93-47bb-824d-b6a1f8e39ff9 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:43.546364 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25769f87-1e93-47bb-824d-b6a1f8e39ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:43.546364 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25769f87-1e93-47bb-824d-b6a1f8e39ff9 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:43.547977 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 369/720] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:43.553956 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44670e2c-e350-47c5-8bd3-c99f8baa4ad8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:43.554644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44670e2c-e350-47c5-8bd3-c99f8baa4ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:43.554644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44670e2c-e350-47c5-8bd3-c99f8baa4ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:43.583010 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-44670e2c-e350-47c5-8bd3-c99f8baa4ad8 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:43.584570 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-44670e2c-e350-47c5-8bd3-c99f8baa4ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:43.692542 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 352/721] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 15:51:43.743525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25769f87-1e93-47bb-824d-b6a1f8e39ff9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:43.905588 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25769f87-1e93-47bb-824d-b6a1f8e39ff9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:43.930750 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:43.958572 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ebdb5-8a67-4e15-9d29-77fcf338f20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:43.959252 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ebdb5-8a67-4e15-9d29-77fcf338f20e None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:43.959535 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ebdb5-8a67-4e15-9d29-77fcf338f20e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:43.959812 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ebdb5-8a67-4e15-9d29-77fcf338f20e None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:43.960547 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30ebdb5-8a67-4e15-9d29-77fcf338f20e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:43.961973 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30ebdb5-8a67-4e15-9d29-77fcf338f20e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:43.986728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 370/722] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 21 15:51:43 2022] POST /identity/v3/services => generated 251 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:51:43.988230 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82072977-8199-42f8-827e-a19b6aaf0690 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:43.988583 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82072977-8199-42f8-827e-a19b6aaf0690 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:43.988872 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82072977-8199-42f8-827e-a19b6aaf0690 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:43.990394 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 353/723] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 15:51:43.996584 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc79d0c-a74e-42a4-8818-1f03997f0389 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:43.997021 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc79d0c-a74e-42a4-8818-1f03997f0389 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:43.997410 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc79d0c-a74e-42a4-8818-1f03997f0389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:44.023119 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0dc79d0c-a74e-42a4-8818-1f03997f0389 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:44.025098 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0dc79d0c-a74e-42a4-8818-1f03997f0389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:44.169266 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 371/724] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:43 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:51:44.203605 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82072977-8199-42f8-827e-a19b6aaf0690 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:44.349980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:44.423222 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82072977-8199-42f8-827e-a19b6aaf0690 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:44.439749 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791df5ba-0842-48d4-9d7b-4955e8a01341 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:44.440095 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791df5ba-0842-48d4-9d7b-4955e8a01341 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:44.440322 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791df5ba-0842-48d4-9d7b-4955e8a01341 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:44.443079 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 372/725] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 15:51:44.449639 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7583ce-bb03-436c-87d6-f61063fb23c7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:44.449911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7583ce-bb03-436c-87d6-f61063fb23c7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:44.450195 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7583ce-bb03-436c-87d6-f61063fb23c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:44.478517 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1c7583ce-bb03-436c-87d6-f61063fb23c7 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:44.479824 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1c7583ce-bb03-436c-87d6-f61063fb23c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:44.488353 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6699c51f-218d-4509-ae64-e9897f905816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:44.489167 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699c51f-218d-4509-ae64-e9897f905816 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:44.489453 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699c51f-218d-4509-ae64-e9897f905816 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:44.489764 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699c51f-218d-4509-ae64-e9897f905816 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:44.512447 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6699c51f-218d-4509-ae64-e9897f905816 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:44.527043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-6699c51f-218d-4509-ae64-e9897f905816 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:51:44.527043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:44.527043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:44.527043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:44.527043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:44.527043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:44.527043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:44.527043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:44.527043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:44.527043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:44.527043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:44.527043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:44.527043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:44.527043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:44.527043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:44.527043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:44.528139 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:44.528139 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:44.528139 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:44.528139 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:44.528139 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:44.528139 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:44.528139 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:44.528139 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:44.528139 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:44.528139 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:44.528139 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:44.528139 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:44.528139 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:44.528139 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:44.528139 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:44.528139 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:44.529061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:44.529061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:44.529061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:44.529061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:44.529061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:44.529061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:44.529061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:44.529061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:44.529061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:44.529061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:44.529061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:51:44.529061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:44.531666 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 354/726] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:51:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:44.542631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6699c51f-218d-4509-ae64-e9897f905816 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:44.662755 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 373/727] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:44 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 15:51:44.709318 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7583ce-bb03-436c-87d6-f61063fb23c7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:44.778349 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:44.805042 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6699c51f-218d-4509-ae64-e9897f905816 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:44.853959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:44.896755 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ad74f-3fcc-4f57-a04b-65280fec7544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:44.897679 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ad74f-3fcc-4f57-a04b-65280fec7544 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:44.898033 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ad74f-3fcc-4f57-a04b-65280fec7544 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:44.898317 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ad74f-3fcc-4f57-a04b-65280fec7544 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:44.899100 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ad74f-3fcc-4f57-a04b-65280fec7544 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:44.899413 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ad74f-3fcc-4f57-a04b-65280fec7544 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:44.900848 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ad74f-3fcc-4f57-a04b-65280fec7544 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:44.917313 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 355/728] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:51:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:44.918962 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3a8af-e631-401d-b1e4-6ce69ed246fa None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:44.919261 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3a8af-e631-401d-b1e4-6ce69ed246fa None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:44.919551 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3a8af-e631-401d-b1e4-6ce69ed246fa None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:44.921109 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 356/729] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:44.926720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc3a8af-e631-401d-b1e4-6ce69ed246fa None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:44.941385 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7583ce-bb03-436c-87d6-f61063fb23c7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:44.980944 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70009c4e-8112-43c0-8362-f3122e90eaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:44.981491 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70009c4e-8112-43c0-8362-f3122e90eaec None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:44.981815 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70009c4e-8112-43c0-8362-f3122e90eaec None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:44.982163 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70009c4e-8112-43c0-8362-f3122e90eaec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:44.993792 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70009c4e-8112-43c0-8362-f3122e90eaec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:45.002848 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-70009c4e-8112-43c0-8362-f3122e90eaec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:51:45.002848 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:45.002848 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:45.002848 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:45.002848 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:45.002848 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:45.002848 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:45.002848 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:45.002848 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:45.002848 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:45.002848 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:45.002848 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:45.002848 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:45.002848 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:45.002848 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:45.002848 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:45.003897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:45.003897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:45.003897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:45.003897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:45.003897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:45.003897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:45.003897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:45.003897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:45.003897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:45.003897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:45.003897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:45.003897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:45.003897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:45.003897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:45.003897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:45.003897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:45.004770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:45.004770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:45.004770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:45.004770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:45.004770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:45.004770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:45.004770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:45.004770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:45.004770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:45.004770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:45.004770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:51:45.004770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:45.004770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 374/730] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:51:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:51:45.006057 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cef4dd-46f9-4100-b073-becfdf99516c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:45.006159 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cef4dd-46f9-4100-b073-becfdf99516c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:45.006462 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cef4dd-46f9-4100-b073-becfdf99516c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:45.028358 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-83cef4dd-46f9-4100-b073-becfdf99516c None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:45.029753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-83cef4dd-46f9-4100-b073-becfdf99516c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:45.089913 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc3a8af-e631-401d-b1e4-6ce69ed246fa None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:45.115524 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 375/731] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:45 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 15:51:45.117701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cef4dd-46f9-4100-b073-becfdf99516c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:45.137132 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:45.144192 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2188c77a-87d8-48e2-96ba-a48bb0e19b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:45.145002 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188c77a-87d8-48e2-96ba-a48bb0e19b45 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:45.145293 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188c77a-87d8-48e2-96ba-a48bb0e19b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:45.145619 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188c77a-87d8-48e2-96ba-a48bb0e19b45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:45.157484 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2188c77a-87d8-48e2-96ba-a48bb0e19b45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:45.159259 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2188c77a-87d8-48e2-96ba-a48bb0e19b45 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:45.180421 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-2188c77a-87d8-48e2-96ba-a48bb0e19b45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:45.180421 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:45.180421 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:45.180421 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:45.180421 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:45.180421 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:45.180421 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:45.180421 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:45.180421 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:45.180421 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:45.180421 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:45.180421 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:45.180421 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:45.180421 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:45.180421 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:45.181784 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:45.181784 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:45.181784 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:45.181784 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:45.181784 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:45.181784 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:45.181784 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:45.181784 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:45.181784 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:45.181784 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:45.181784 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:45.181784 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:45.181784 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:45.181784 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:45.181784 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:45.181784 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:45.181784 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:45.182950 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:45.182950 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:45.182950 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:45.182950 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:45.182950 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 357/732] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:45.183341 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c5448-a8b5-43c9-8ba9-6d26721e5abc None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:45.183341 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c5448-a8b5-43c9-8ba9-6d26721e5abc None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:45.183668 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c5448-a8b5-43c9-8ba9-6d26721e5abc None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:45.185563 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 358/733] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:45.191335 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c5448-a8b5-43c9-8ba9-6d26721e5abc None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:45.230908 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:45.261173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cef4dd-46f9-4100-b073-becfdf99516c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:45.300172 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb2a18-fac2-4386-86bc-fb0da3770bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:45.300849 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb2a18-fac2-4386-86bc-fb0da3770bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:45.301136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb2a18-fac2-4386-86bc-fb0da3770bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:45.301418 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb2a18-fac2-4386-86bc-fb0da3770bd2 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:45.302213 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdb2a18-fac2-4386-86bc-fb0da3770bd2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:45.302513 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdb2a18-fac2-4386-86bc-fb0da3770bd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:45.303922 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdb2a18-fac2-4386-86bc-fb0da3770bd2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:45.318534 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 376/734] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:51:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:45.318534 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb2a18-fac2-4386-86bc-fb0da3770bd2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:45.320543 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c5448-a8b5-43c9-8ba9-6d26721e5abc None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:45.358074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756af7f-3207-4eda-aa18-15964238b544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:45.359357 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756af7f-3207-4eda-aa18-15964238b544 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:45.359715 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756af7f-3207-4eda-aa18-15964238b544 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:45.360087 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756af7f-3207-4eda-aa18-15964238b544 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:45.360896 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7756af7f-3207-4eda-aa18-15964238b544 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:45.362399 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7756af7f-3207-4eda-aa18-15964238b544 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:45.382268 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 359/735] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 21 15:51:45 2022] POST /identity/v3/services => generated 240 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:51:45.383322 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758615b-1f0a-4baa-93aa-cf63fed38e0b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:45.383651 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758615b-1f0a-4baa-93aa-cf63fed38e0b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:45.383888 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758615b-1f0a-4baa-93aa-cf63fed38e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:45.406765 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:45.421044 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7758615b-1f0a-4baa-93aa-cf63fed38e0b None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:45.424125 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7758615b-1f0a-4baa-93aa-cf63fed38e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:45.455436 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb2a18-fac2-4386-86bc-fb0da3770bd2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:45.511966 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c642aa5-a6b6-4369-aead-ca9c5fda718c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:45.512698 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c642aa5-a6b6-4369-aead-ca9c5fda718c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:45.513056 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c642aa5-a6b6-4369-aead-ca9c5fda718c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:45.513401 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c642aa5-a6b6-4369-aead-ca9c5fda718c None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:45.514267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c642aa5-a6b6-4369-aead-ca9c5fda718c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:45.514651 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c642aa5-a6b6-4369-aead-ca9c5fda718c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:45.516424 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c642aa5-a6b6-4369-aead-ca9c5fda718c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:45.530049 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 360/736] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:51:45.533617 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 377/737] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:45.536197 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758615b-1f0a-4baa-93aa-cf63fed38e0b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:45.537413 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c642aa5-a6b6-4369-aead-ca9c5fda718c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:45.583496 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:45.674888 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c642aa5-a6b6-4369-aead-ca9c5fda718c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:45.709083 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758615b-1f0a-4baa-93aa-cf63fed38e0b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:45.725402 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:45.733988 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407efa93-fb0b-4b53-84b2-f2fb36edb6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:45.734811 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407efa93-fb0b-4b53-84b2-f2fb36edb6a3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:45.735094 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407efa93-fb0b-4b53-84b2-f2fb36edb6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:45.735942 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407efa93-fb0b-4b53-84b2-f2fb36edb6a3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:45.747411 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407efa93-fb0b-4b53-84b2-f2fb36edb6a3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:45.749190 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407efa93-fb0b-4b53-84b2-f2fb36edb6a3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:45.755114 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422869b-c7a3-4679-9cd2-eb33fad14a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:45.755114 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422869b-c7a3-4679-9cd2-eb33fad14a82 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:45.755114 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422869b-c7a3-4679-9cd2-eb33fad14a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:45.755472 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422869b-c7a3-4679-9cd2-eb33fad14a82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:45.770686 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c422869b-c7a3-4679-9cd2-eb33fad14a82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:45.771284 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-407efa93-fb0b-4b53-84b2-f2fb36edb6a3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 15:51:45.771284 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:45.771284 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:45.771284 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:45.771284 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:45.771284 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:45.771284 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:45.771284 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:45.771284 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:45.771284 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:45.771284 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:45.771284 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:45.771284 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:51:45.771284 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:51:45.771284 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:51:45.771284 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:51:45.772226 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:45.772226 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:45.772226 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:51:45.772226 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:45.772226 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:51:45.772226 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:45.772226 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:45.772226 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:45.772226 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:45.772226 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:45.772226 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:45.772226 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:45.772226 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:45.772226 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:45.772226 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:45.772226 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:45.772226 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:45.772226 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:45.773205 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:45.773205 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:45.773205 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:51:45.773205 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:51:45.773205 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:51:45.773205 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:51:45.773205 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:51:45.773205 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:51:45.773205 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 15:51:45.773205 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:45.773781 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c422869b-c7a3-4679-9cd2-eb33fad14a82 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:45.773942 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 378/738] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:51:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:45.775664 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407efa93-fb0b-4b53-84b2-f2fb36edb6a3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:45.807818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-c422869b-c7a3-4679-9cd2-eb33fad14a82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:45.807818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:45.807818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:45.807818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:45.807818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:45.807818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:45.807818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:45.807818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:45.807818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:45.807818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:45.807818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:45.807818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:45.807818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:45.807818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:45.807818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:45.810001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:45.810001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:45.810001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:45.810001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:45.810001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:45.810001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:45.810001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:45.810001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:45.810001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:45.810001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:45.810001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:45.810001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:45.810001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:45.810001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:45.810001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:45.810001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:45.810001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:45.812356 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:45.812356 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:45.812356 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:45.812356 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:45.812356 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 361/739] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:45.812356 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422869b-c7a3-4679-9cd2-eb33fad14a82 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:45.920006 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407efa93-fb0b-4b53-84b2-f2fb36edb6a3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:45.926414 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:45.965538 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422869b-c7a3-4679-9cd2-eb33fad14a82 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:45.979381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913bad72-8ef5-40bd-82d4-51b7317b909d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:45.980553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913bad72-8ef5-40bd-82d4-51b7317b909d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:45.980842 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913bad72-8ef5-40bd-82d4-51b7317b909d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:45.981120 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913bad72-8ef5-40bd-82d4-51b7317b909d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:45.996203 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913bad72-8ef5-40bd-82d4-51b7317b909d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:46.006645 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-913bad72-8ef5-40bd-82d4-51b7317b909d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:46.006645 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:46.006645 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:46.006645 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:46.006645 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:46.006645 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:46.006645 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:46.006645 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:46.006645 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:46.006645 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:46.006645 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:46.006645 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:46.006645 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:46.006645 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:46.006645 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:46.006645 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:46.007690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:46.007690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:46.007690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:46.007690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:46.007690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:46.007690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:46.007690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:46.007690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:46.007690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:46.007690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:46.007690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:46.007690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:46.007690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:46.007690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:46.007690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:46.007690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:46.008571 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:46.008571 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:46.008571 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:46.008571 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:46.008571 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:46.008571 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:46.008571 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:46.008571 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:46.008571 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:46.008571 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:46.008571 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:46.008571 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:46.008571 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 379/740] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 15:51:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:46.009346 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913bad72-8ef5-40bd-82d4-51b7317b909d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:46.011335 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d4b6c0-c4c5-41fd-b735-db5a710db262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:46.013151 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4b6c0-c4c5-41fd-b735-db5a710db262 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:46.014004 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4b6c0-c4c5-41fd-b735-db5a710db262 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:46.014801 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4b6c0-c4c5-41fd-b735-db5a710db262 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:46.017047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d4b6c0-c4c5-41fd-b735-db5a710db262 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:46.017952 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d4b6c0-c4c5-41fd-b735-db5a710db262 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:46.020959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d4b6c0-c4c5-41fd-b735-db5a710db262 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:46.055786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 362/741] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Jul 21 15:51:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:46.057170 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d4b6c0-c4c5-41fd-b735-db5a710db262 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:46.114211 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:46.135613 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913bad72-8ef5-40bd-82d4-51b7317b909d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:46.184554 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc68e86-5667-4a8e-9261-098070fd5dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:46.185234 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc68e86-5667-4a8e-9261-098070fd5dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:46.185515 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc68e86-5667-4a8e-9261-098070fd5dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:46.185961 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc68e86-5667-4a8e-9261-098070fd5dd0 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:46.186902 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc68e86-5667-4a8e-9261-098070fd5dd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:46.187213 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc68e86-5667-4a8e-9261-098070fd5dd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:46.188908 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc68e86-5667-4a8e-9261-098070fd5dd0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:46.206217 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 380/742] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:46.207152 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc68e86-5667-4a8e-9261-098070fd5dd0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:46.207379 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d4b6c0-c4c5-41fd-b735-db5a710db262 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:46.253273 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c2f56-dbb3-4c16-81d5-5e09c2a96955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:46.254498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c2f56-dbb3-4c16-81d5-5e09c2a96955 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:46.255075 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c2f56-dbb3-4c16-81d5-5e09c2a96955 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:46.255567 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c2f56-dbb3-4c16-81d5-5e09c2a96955 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:46.258098 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00c2f56-dbb3-4c16-81d5-5e09c2a96955 None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:46.258936 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00c2f56-dbb3-4c16-81d5-5e09c2a96955 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:46.261767 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00c2f56-dbb3-4c16-81d5-5e09c2a96955 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:46.267528 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:46.278494 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 363/743] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 15:51:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:46.280312 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c2f56-dbb3-4c16-81d5-5e09c2a96955 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:46.349516 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc68e86-5667-4a8e-9261-098070fd5dd0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:46.366833 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:46.399479 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35a31e-13c8-4866-bd29-7b8ae459ec56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:46.400179 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35a31e-13c8-4866-bd29-7b8ae459ec56 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:46.400468 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35a31e-13c8-4866-bd29-7b8ae459ec56 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:46.400743 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35a31e-13c8-4866-bd29-7b8ae459ec56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:46.416799 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c2f56-dbb3-4c16-81d5-5e09c2a96955 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:46.455646 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1da9f0-43ce-4caf-bdb7-dff615b9924d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:46.456862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1da9f0-43ce-4caf-bdb7-dff615b9924d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:46.457376 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1da9f0-43ce-4caf-bdb7-dff615b9924d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:46.457912 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1da9f0-43ce-4caf-bdb7-dff615b9924d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:46.466816 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1da9f0-43ce-4caf-bdb7-dff615b9924d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:46.469353 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1da9f0-43ce-4caf-bdb7-dff615b9924d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:46.479296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d35a31e-13c8-4866-bd29-7b8ae459ec56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:46.480760 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d35a31e-13c8-4866-bd29-7b8ae459ec56 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:46.485906 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-5f1da9f0-43ce-4caf-bdb7-dff615b9924d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 15:51:46.485906 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:46.485906 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:46.485906 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:46.485906 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:46.485906 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:46.485906 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:46.485906 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:46.485906 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:46.485906 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:46.485906 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:46.485906 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:46.485906 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:51:46.485906 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:51:46.485906 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:51:46.485906 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:51:46.486862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:46.486862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:46.486862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:51:46.486862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:46.486862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:51:46.486862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:46.486862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:46.486862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:46.486862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:46.486862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:46.486862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:46.486862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:46.486862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:46.486862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:46.486862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:46.486862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:46.486862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:46.486862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:46.487843 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:46.487843 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:46.487843 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:51:46.487843 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:51:46.487843 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:51:46.487843 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:51:46.487843 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:51:46.487843 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:51:46.487843 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 15:51:46.487843 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:46.490602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 364/744] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jul 21 15:51:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:46.491792 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1da9f0-43ce-4caf-bdb7-dff615b9924d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:46.573545 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:46.593613 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 381/745] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:51:46 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 15:51:46.593613 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35a31e-13c8-4866-bd29-7b8ae459ec56 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:46.603996 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:46.633253 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1da9f0-43ce-4caf-bdb7-dff615b9924d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:46.688452 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d482256-d8f7-427f-8b9b-5ae8001045a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:46.689708 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d482256-d8f7-427f-8b9b-5ae8001045a2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:46.690300 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d482256-d8f7-427f-8b9b-5ae8001045a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:46.690852 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d482256-d8f7-427f-8b9b-5ae8001045a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:46.703719 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35a31e-13c8-4866-bd29-7b8ae459ec56 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:46.710061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d482256-d8f7-427f-8b9b-5ae8001045a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:46.712672 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d482256-d8f7-427f-8b9b-5ae8001045a2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:46.734359 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-5d482256-d8f7-427f-8b9b-5ae8001045a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:46.734359 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:46.734359 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:46.734359 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:46.734359 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:46.734359 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:46.734359 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:46.734359 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:46.734359 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:46.734359 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:46.734359 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:46.734359 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:46.734359 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:46.734359 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:46.734359 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:46.735344 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:46.735344 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:46.735344 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:46.735344 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:46.735344 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:46.735344 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:46.735344 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:46.735344 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:46.735344 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:46.735344 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:46.735344 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:46.735344 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:46.735344 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:46.735344 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:46.735344 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:46.735344 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:46.735344 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:46.736253 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:46.736253 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:46.736253 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:46.736253 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:46.736253 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 365/746] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:46.736533 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d482256-d8f7-427f-8b9b-5ae8001045a2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:46.760308 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e176fa-51be-4b6f-a71c-e4355ce05afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:46.761111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e176fa-51be-4b6f-a71c-e4355ce05afc None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:46.762211 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e176fa-51be-4b6f-a71c-e4355ce05afc None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:46.762211 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e176fa-51be-4b6f-a71c-e4355ce05afc None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:46.763032 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e176fa-51be-4b6f-a71c-e4355ce05afc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:46.763032 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e176fa-51be-4b6f-a71c-e4355ce05afc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:46.764688 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e176fa-51be-4b6f-a71c-e4355ce05afc None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:46.786904 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 382/747] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jul 21 15:51:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:46.788608 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c38ccf-4a80-40a5-9487-8fdf8a7e248a None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:46.788895 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c38ccf-4a80-40a5-9487-8fdf8a7e248a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:46.789170 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c38ccf-4a80-40a5-9487-8fdf8a7e248a None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:46.790628 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 383/748] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:46.791364 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c38ccf-4a80-40a5-9487-8fdf8a7e248a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:46.868621 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:46.882379 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d482256-d8f7-427f-8b9b-5ae8001045a2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:46.898601 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:46.913239 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c38ccf-4a80-40a5-9487-8fdf8a7e248a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:46.932339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf9d86a-53d0-45aa-8712-1c9e199c2b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:46.933142 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9d86a-53d0-45aa-8712-1c9e199c2b12 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:46.933433 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9d86a-53d0-45aa-8712-1c9e199c2b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:46.933743 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9d86a-53d0-45aa-8712-1c9e199c2b12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:46.945232 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf9d86a-53d0-45aa-8712-1c9e199c2b12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:46.946848 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf9d86a-53d0-45aa-8712-1c9e199c2b12 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:46.958420 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03cff9-e5cf-4aeb-bb9f-f9895fd14987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:46.959091 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03cff9-e5cf-4aeb-bb9f-f9895fd14987 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:46.959373 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03cff9-e5cf-4aeb-bb9f-f9895fd14987 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:46.959647 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03cff9-e5cf-4aeb-bb9f-f9895fd14987 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:46.960377 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b03cff9-e5cf-4aeb-bb9f-f9895fd14987 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:46.960688 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b03cff9-e5cf-4aeb-bb9f-f9895fd14987 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:46.962103 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b03cff9-e5cf-4aeb-bb9f-f9895fd14987 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:46.966265 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-0cf9d86a-53d0-45aa-8712-1c9e199c2b12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:46.966265 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:46.966265 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:46.966265 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:46.966265 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:46.966265 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:46.966265 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:46.966265 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:46.966265 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:46.966265 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:46.966265 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:46.966265 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:46.966265 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:46.966265 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:46.966265 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:46.967177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:46.967177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:46.967177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:46.967177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:46.967177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:46.967177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:46.967177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:46.967177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:46.967177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:46.967177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:46.967177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:46.967177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:46.967177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:46.967177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:46.967177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:46.967177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:46.967177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:46.974764 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:46.974764 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:46.974764 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:46.974764 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:46.974764 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 366/749] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:46.974764 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf9d86a-53d0-45aa-8712-1c9e199c2b12 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:46.975439 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 384/750] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:46.977088 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027eb3b5-09ae-4467-9fa0-ca434982e391 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:46.977946 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027eb3b5-09ae-4467-9fa0-ca434982e391 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:46.978236 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027eb3b5-09ae-4467-9fa0-ca434982e391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:46.997729 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-027eb3b5-09ae-4467-9fa0-ca434982e391 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:46.999996 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-027eb3b5-09ae-4467-9fa0-ca434982e391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:47.097644 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 385/751] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:51:47.097881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf9d86a-53d0-45aa-8712-1c9e199c2b12 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:47.098001 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027eb3b5-09ae-4467-9fa0-ca434982e391 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:47.144882 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3106e560-8e91-4d5e-860c-6ce9d88a697f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:47.145688 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106e560-8e91-4d5e-860c-6ce9d88a697f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:47.145973 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106e560-8e91-4d5e-860c-6ce9d88a697f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:47.146247 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106e560-8e91-4d5e-860c-6ce9d88a697f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:47.162061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3106e560-8e91-4d5e-860c-6ce9d88a697f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:47.163268 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:47.164480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3106e560-8e91-4d5e-860c-6ce9d88a697f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:47.183725 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-3106e560-8e91-4d5e-860c-6ce9d88a697f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:47.183725 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:47.183725 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:47.183725 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:47.183725 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:47.183725 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:47.183725 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:47.183725 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:47.183725 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:47.183725 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:47.183725 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:47.183725 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:47.183725 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:47.183725 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:47.183725 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:47.184738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:47.184738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:47.184738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:47.184738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:47.184738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:47.184738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:47.184738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:47.184738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:47.184738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:47.184738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:47.184738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:47.184738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:47.184738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:47.184738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:47.184738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:47.184738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:47.184738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:47.186476 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:47.186476 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:47.186476 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:47.186476 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:47.186476 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 367/752] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:47.186476 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3106e560-8e91-4d5e-860c-6ce9d88a697f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:47.204701 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:47.221461 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027eb3b5-09ae-4467-9fa0-ca434982e391 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:47.265261 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee07a1-e166-4698-bc38-de3d0b51f2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:47.265974 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee07a1-e166-4698-bc38-de3d0b51f2a2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:47.266263 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee07a1-e166-4698-bc38-de3d0b51f2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:47.266546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee07a1-e166-4698-bc38-de3d0b51f2a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:47.267292 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee07a1-e166-4698-bc38-de3d0b51f2a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:47.267609 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee07a1-e166-4698-bc38-de3d0b51f2a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:47.269543 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee07a1-e166-4698-bc38-de3d0b51f2a2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:47.282408 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 386/753] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:47.283088 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee07a1-e166-4698-bc38-de3d0b51f2a2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:47.329279 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3106e560-8e91-4d5e-860c-6ce9d88a697f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:47.380986 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a79f5-d6da-4d59-ba1e-eea777749e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:47.381960 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a79f5-d6da-4d59-ba1e-eea777749e15 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:47.382314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a79f5-d6da-4d59-ba1e-eea777749e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:47.382660 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a79f5-d6da-4d59-ba1e-eea777749e15 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:47.390854 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a79f5-d6da-4d59-ba1e-eea777749e15 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:47.392324 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a79f5-d6da-4d59-ba1e-eea777749e15 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:47.412215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-901a79f5-d6da-4d59-ba1e-eea777749e15 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 15:51:47.412215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:47.412215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:47.412215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:47.412215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:47.412215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:47.412215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:47.412215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:47.412215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:47.412215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:47.412215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:47.412215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:47.412215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:51:47.412215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:51:47.412215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:51:47.412215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:51:47.413236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:47.413236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:47.413236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:51:47.413236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:47.413236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:51:47.413236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:47.413236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:47.413236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:47.413236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:47.413236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:47.413236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:47.413236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:47.413236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:47.413236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:47.413236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:47.413236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:47.413236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:47.413236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:47.414464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:47.414464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:47.414464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:51:47.414464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:51:47.414464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:51:47.414464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:51:47.414464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:51:47.414464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:51:47.414464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 15:51:47.414464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:47.414464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 368/754] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jul 21 15:51:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:47.414464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a79f5-d6da-4d59-ba1e-eea777749e15 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:47.428291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee07a1-e166-4698-bc38-de3d0b51f2a2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:47.456927 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:47.479098 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172961b9-3a93-4fdd-89f8-85c284500a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:47.479925 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172961b9-3a93-4fdd-89f8-85c284500a3d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:47.480212 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172961b9-3a93-4fdd-89f8-85c284500a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:47.480489 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172961b9-3a93-4fdd-89f8-85c284500a3d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:47.481172 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172961b9-3a93-4fdd-89f8-85c284500a3d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:47.482613 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172961b9-3a93-4fdd-89f8-85c284500a3d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:47.496035 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-172961b9-3a93-4fdd-89f8-85c284500a3d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 15:51:47.496035 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:47.496035 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 15:51:47.496035 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 15:51:47.496035 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 15:51:47.496035 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 15:51:47.496035 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 15:51:47.496035 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:51:47.496035 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 15:51:47.496035 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:47.496035 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 15:51:47.496035 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:47.496035 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:47.496035 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:47.496035 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:47.496035 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:47.497080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:47.497080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:47.497080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:47.497080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:47.497080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:47.497080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:47.497080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:47.497080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 15:51:47.497080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 15:51:47.497080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 15:51:47.497080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 15:51:47.497080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:47.497080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:47.497080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:47.497080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:47.497080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:47.498045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:47.498045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:47.498045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:47.498045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:47.498045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:47.498045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:47.498045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:47.498045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:47.498045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:47.498045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:47.498045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:47.498045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 15:51:47.498045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:51:47.498045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 15:51:47.498045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:47.498045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 387/755] 10.222.0.52 () {56 vars in 1129 bytes} [Thu Jul 21 15:51:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:47.500203 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172961b9-3a93-4fdd-89f8-85c284500a3d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:47.514064 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:47.562767 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a79f5-d6da-4d59-ba1e-eea777749e15 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:47.602595 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc30d966-413f-463a-916b-496b5dd3c2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:47.603376 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc30d966-413f-463a-916b-496b5dd3c2c6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:47.603657 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc30d966-413f-463a-916b-496b5dd3c2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:47.603930 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc30d966-413f-463a-916b-496b5dd3c2c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:47.604662 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc30d966-413f-463a-916b-496b5dd3c2c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:47.604969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc30d966-413f-463a-916b-496b5dd3c2c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:47.606489 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc30d966-413f-463a-916b-496b5dd3c2c6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:47.622750 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172961b9-3a93-4fdd-89f8-85c284500a3d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:47.626476 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 369/756] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:47.628017 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc30d966-413f-463a-916b-496b5dd3c2c6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:47.666185 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c79b4-7cf0-4fd3-903e-a83d6d2059f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:47.666881 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c79b4-7cf0-4fd3-903e-a83d6d2059f3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:47.667166 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c79b4-7cf0-4fd3-903e-a83d6d2059f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:47.667444 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c79b4-7cf0-4fd3-903e-a83d6d2059f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:47.675236 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c79b4-7cf0-4fd3-903e-a83d6d2059f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:47.676742 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c79b4-7cf0-4fd3-903e-a83d6d2059f3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:47.692605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-cd6c79b4-7cf0-4fd3-903e-a83d6d2059f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:47.692605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:47.692605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:47.692605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:47.692605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:47.692605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:47.692605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:47.692605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:47.692605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:47.692605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:47.692605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:47.692605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:47.692605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:51:47.692605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:51:47.692605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:51:47.693647 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:51:47.693647 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:47.693647 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:47.693647 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:47.693647 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:47.693647 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:47.693647 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:47.693647 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:47.693647 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:47.693647 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:47.693647 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:47.693647 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:47.693647 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:47.693647 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:47.693647 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:47.693647 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:47.693647 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:47.694570 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:51:47.694570 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:51:47.694570 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:51:47.694570 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:51:47.694570 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:51:47.694570 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:51:47.694570 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:47.694570 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:47.694570 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 388/757] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:51:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:47.695283 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c79b4-7cf0-4fd3-903e-a83d6d2059f3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:47.740432 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:47.749849 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc30d966-413f-463a-916b-496b5dd3c2c6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:47.789231 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:47.796130 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c3cb0f-f768-4e5d-a439-88bdda4fee9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:47.797329 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3cb0f-f768-4e5d-a439-88bdda4fee9b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:47.797631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3cb0f-f768-4e5d-a439-88bdda4fee9b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:47.797914 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3cb0f-f768-4e5d-a439-88bdda4fee9b None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:47.798775 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c3cb0f-f768-4e5d-a439-88bdda4fee9b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:47.799087 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c3cb0f-f768-4e5d-a439-88bdda4fee9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:47.800541 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c3cb0f-f768-4e5d-a439-88bdda4fee9b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:47.803966 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c79b4-7cf0-4fd3-903e-a83d6d2059f3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:47.818983 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 370/758] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Jul 21 15:51:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:47.820328 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283c746-bcd4-46b6-a5b2-af9e9724aa9a None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:47.820621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283c746-bcd4-46b6-a5b2-af9e9724aa9a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:47.820902 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283c746-bcd4-46b6-a5b2-af9e9724aa9a None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:47.822407 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 371/759] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:47.823083 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a283c746-bcd4-46b6-a5b2-af9e9724aa9a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:47.848977 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b05b23-23c5-4e04-a4fe-74f2cb25adc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:47.849673 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b05b23-23c5-4e04-a4fe-74f2cb25adc6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:47.849961 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b05b23-23c5-4e04-a4fe-74f2cb25adc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:47.850236 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b05b23-23c5-4e04-a4fe-74f2cb25adc6 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:47.850962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b05b23-23c5-4e04-a4fe-74f2cb25adc6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:47.851269 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b05b23-23c5-4e04-a4fe-74f2cb25adc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:47.852670 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b05b23-23c5-4e04-a4fe-74f2cb25adc6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:47.857695 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 389/760] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Jul 21 15:51:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:47.858502 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b05b23-23c5-4e04-a4fe-74f2cb25adc6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:47.945664 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a283c746-bcd4-46b6-a5b2-af9e9724aa9a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:47.970792 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b05b23-23c5-4e04-a4fe-74f2cb25adc6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:47.979050 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:47.985422 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4968ef8-8402-48e9-bd1a-cf857ee7e747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:47.986249 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4968ef8-8402-48e9-bd1a-cf857ee7e747 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:47.986530 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4968ef8-8402-48e9-bd1a-cf857ee7e747 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:47.986806 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4968ef8-8402-48e9-bd1a-cf857ee7e747 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:47.993828 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4968ef8-8402-48e9-bd1a-cf857ee7e747 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:47.995296 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4968ef8-8402-48e9-bd1a-cf857ee7e747 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:48.009594 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b4968ef8-8402-48e9-bd1a-cf857ee7e747 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:48.009594 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:48.009594 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:48.009594 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:48.009594 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:48.009594 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:48.009594 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:48.009594 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:48.009594 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:48.009594 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:48.009594 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:48.009594 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:48.009594 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:51:48.009594 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:51:48.009594 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:51:48.010779 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:51:48.010779 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:48.010779 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:48.010779 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:48.010779 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:48.010779 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:48.010779 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:48.010779 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:48.010779 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:48.010779 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:48.010779 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:48.010779 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:48.010779 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:48.010779 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:48.010779 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:48.010779 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:48.010779 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:48.011720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:51:48.011720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:51:48.011720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:51:48.011720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:51:48.011720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:51:48.011720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:51:48.011720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:48.011720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:48.011720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 372/761] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:51:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:48.011720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4968ef8-8402-48e9-bd1a-cf857ee7e747 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:48.013099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a5550-0a2a-4d40-a20d-ef73ca1f2338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:48.013793 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a5550-0a2a-4d40-a20d-ef73ca1f2338 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:48.014077 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a5550-0a2a-4d40-a20d-ef73ca1f2338 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:48.014353 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a5550-0a2a-4d40-a20d-ef73ca1f2338 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:48.015230 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770a5550-0a2a-4d40-a20d-ef73ca1f2338 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:48.015489 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770a5550-0a2a-4d40-a20d-ef73ca1f2338 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:48.016949 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770a5550-0a2a-4d40-a20d-ef73ca1f2338 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:48.029533 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 390/762] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:51:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:48.031364 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b091a-042f-448e-9a05-50a338cf2e03 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:48.031652 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b091a-042f-448e-9a05-50a338cf2e03 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:48.031931 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b091a-042f-448e-9a05-50a338cf2e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:48.049414 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-401b091a-042f-448e-9a05-50a338cf2e03 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:48.050642 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-401b091a-042f-448e-9a05-50a338cf2e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:48.083565 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:48.125438 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 391/763] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:51:48.127323 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b091a-042f-448e-9a05-50a338cf2e03 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:48.152021 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4968ef8-8402-48e9-bd1a-cf857ee7e747 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:48.189033 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386f5fe-2c2c-4b4d-9a09-e3b9865f4cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:48.189721 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386f5fe-2c2c-4b4d-9a09-e3b9865f4cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:48.190006 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386f5fe-2c2c-4b4d-9a09-e3b9865f4cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:48.190276 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386f5fe-2c2c-4b4d-9a09-e3b9865f4cf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:48.240579 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:48.248978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b091a-042f-448e-9a05-50a338cf2e03 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:48.267907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9386f5fe-2c2c-4b4d-9a09-e3b9865f4cf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:48.269480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9386f5fe-2c2c-4b4d-9a09-e3b9865f4cf6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:48.288050 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c997a-5e8d-41a2-b734-4d9d7f16af1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:48.288786 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c997a-5e8d-41a2-b734-4d9d7f16af1c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:48.289075 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c997a-5e8d-41a2-b734-4d9d7f16af1c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:48.289376 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c997a-5e8d-41a2-b734-4d9d7f16af1c None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:48.290135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1c997a-5e8d-41a2-b734-4d9d7f16af1c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:48.290451 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1c997a-5e8d-41a2-b734-4d9d7f16af1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:48.291855 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1c997a-5e8d-41a2-b734-4d9d7f16af1c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:48.298085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 392/764] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Jul 21 15:51:48 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:48.298898 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c997a-5e8d-41a2-b734-4d9d7f16af1c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:48.319856 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:48.376292 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 373/765] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:51:48 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 15:51:48.377254 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386f5fe-2c2c-4b4d-9a09-e3b9865f4cf6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:48.423634 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c997a-5e8d-41a2-b734-4d9d7f16af1c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:48.466935 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c75fae-78a2-4ed4-914e-5eb3af63221a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:48.467598 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c75fae-78a2-4ed4-914e-5eb3af63221a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:48.468746 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c75fae-78a2-4ed4-914e-5eb3af63221a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:48.468886 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c75fae-78a2-4ed4-914e-5eb3af63221a None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:48.469569 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c75fae-78a2-4ed4-914e-5eb3af63221a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:48.469908 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c75fae-78a2-4ed4-914e-5eb3af63221a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:48.471381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c75fae-78a2-4ed4-914e-5eb3af63221a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:48.484131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 393/766] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:51:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:48.484960 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c75fae-78a2-4ed4-914e-5eb3af63221a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:48.486545 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386f5fe-2c2c-4b4d-9a09-e3b9865f4cf6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:48.508065 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:48.524856 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05714f9-f86f-4288-aa23-339cdad3bf09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:48.525513 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05714f9-f86f-4288-aa23-339cdad3bf09 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:48.525833 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05714f9-f86f-4288-aa23-339cdad3bf09 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:48.526109 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05714f9-f86f-4288-aa23-339cdad3bf09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:48.526947 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05714f9-f86f-4288-aa23-339cdad3bf09 None admin] RBAC: Adding query filter params (role.id=475521b8a0174037b162f92022aea1a1, scope.project.id=bb1ac00308714c39a79aae09fbb73021, user.id=f5bfae2cb30843b1b1a0343efef38638) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:48.527263 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05714f9-f86f-4288-aa23-339cdad3bf09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:48.528735 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05714f9-f86f-4288-aa23-339cdad3bf09 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:48.535854 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 374/767] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:51:48 2022] GET /identity/v3/role_assignments?user.id=f5bfae2cb30843b1b1a0343efef38638&scope.project.id=bb1ac00308714c39a79aae09fbb73021&role.id=475521b8a0174037b162f92022aea1a1 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:48.536617 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05714f9-f86f-4288-aa23-339cdad3bf09 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:48.556052 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:48.625972 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c75fae-78a2-4ed4-914e-5eb3af63221a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:48.673290 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e016f1c-6817-4320-9989-ee1478229db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:48.674306 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e016f1c-6817-4320-9989-ee1478229db0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:48.674596 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e016f1c-6817-4320-9989-ee1478229db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:48.674877 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e016f1c-6817-4320-9989-ee1478229db0 None admin] PATH_INFO: `/v3/services/network` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:48.675645 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e016f1c-6817-4320-9989-ee1478229db0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:48.675754 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05714f9-f86f-4288-aa23-339cdad3bf09 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:48.677084 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e016f1c-6817-4320-9989-ee1478229db0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:48.691956 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-1e016f1c-6817-4320-9989-ee1478229db0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 15:51:48.691956 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:48.691956 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 15:51:48.691956 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 15:51:48.691956 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 15:51:48.691956 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 15:51:48.691956 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 15:51:48.691956 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:51:48.691956 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 15:51:48.691956 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:48.691956 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 15:51:48.691956 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:48.691956 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:48.691956 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:48.691956 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:48.691956 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:48.691956 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:48.693084 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:48.693084 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:48.693084 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:48.693084 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:48.693084 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:48.693084 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:48.693084 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 15:51:48.693084 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 15:51:48.693084 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 15:51:48.693084 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 15:51:48.693084 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:48.693084 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:48.693084 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:48.693084 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:48.693084 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:48.693084 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:48.693975 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:48.693975 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:48.693975 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:48.693975 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:48.693975 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:48.693975 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:48.693975 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:48.693975 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:48.693975 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:48.693975 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:48.693975 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 15:51:48.693975 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:51:48.693975 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 15:51:48.693975 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:48.698770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 394/768] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:51:48 2022] GET /identity/v3/services/network => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:48.699049 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e016f1c-6817-4320-9989-ee1478229db0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:48.735059 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c184269-1d36-4d19-8c23-151c4dda9d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:48.736269 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c184269-1d36-4d19-8c23-151c4dda9d02 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:48.736666 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c184269-1d36-4d19-8c23-151c4dda9d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:48.737252 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c184269-1d36-4d19-8c23-151c4dda9d02 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:48.738753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c184269-1d36-4d19-8c23-151c4dda9d02 None admin] RBAC: Adding query filter params (interface=public, service_id=82e4a20cd4aa4f90a6abb3161e590a15, region_id=RegionOne) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:48.739287 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c184269-1d36-4d19-8c23-151c4dda9d02 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:48.740892 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c184269-1d36-4d19-8c23-151c4dda9d02 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:48.750696 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 375/769] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Jul 21 15:51:48 2022] GET /identity/v3/endpoints?service_id=82e4a20cd4aa4f90a6abb3161e590a15&interface=public®ion_id=RegionOne => generated 194 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:48.751409 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c184269-1d36-4d19-8c23-151c4dda9d02 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:48.826400 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:48.852004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e016f1c-6817-4320-9989-ee1478229db0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:48.867951 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:48.893188 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c184269-1d36-4d19-8c23-151c4dda9d02 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:48.907601 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f459d293-e71f-4807-a234-0870e1dd6d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:48.908952 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459d293-e71f-4807-a234-0870e1dd6d73 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:48.909248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459d293-e71f-4807-a234-0870e1dd6d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:48.909531 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459d293-e71f-4807-a234-0870e1dd6d73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:48.922426 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f459d293-e71f-4807-a234-0870e1dd6d73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:48.923892 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f459d293-e71f-4807-a234-0870e1dd6d73 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:48.940554 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac78ab80-504b-4961-8e18-b48ad6f54264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:48.941442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78ab80-504b-4961-8e18-b48ad6f54264 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:48.941755 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78ab80-504b-4961-8e18-b48ad6f54264 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:48.942032 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78ab80-504b-4961-8e18-b48ad6f54264 None admin] PATH_INFO: `/v3/projects/bb1ac00308714c39a79aae09fbb73021/users/092553a86f8848dba8c7271f5c0b2c46/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:48.943618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-f459d293-e71f-4807-a234-0870e1dd6d73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:48.943618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:48.943618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:48.943618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:48.943618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:48.943618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:48.943618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:48.943618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:48.943618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:48.943618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:48.943618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:48.943618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:48.943618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:48.943618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:48.943618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:48.944797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:48.944797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:48.944797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:48.944797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:48.944797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:48.944797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:48.944797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:48.944797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:48.944797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:48.944797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:48.944797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:48.944797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:48.944797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:48.944797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:48.944797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:48.944797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:48.944797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:48.946048 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:48.946048 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:48.946048 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:48.946048 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:48.946328 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac78ab80-504b-4961-8e18-b48ad6f54264 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb1ac00308714c39a79aae09fbb73021, user_id=092553a86f8848dba8c7271f5c0b2c46, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:48.946328 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac78ab80-504b-4961-8e18-b48ad6f54264 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:48.947649 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 395/770] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:48.953282 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f459d293-e71f-4807-a234-0870e1dd6d73 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:48.966001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 376/771] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 15:51:48 2022] PUT /identity/v3/projects/bb1ac00308714c39a79aae09fbb73021/users/092553a86f8848dba8c7271f5c0b2c46/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 15:51:48.967383 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac78ab80-504b-4961-8e18-b48ad6f54264 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:49.088677 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f459d293-e71f-4807-a234-0870e1dd6d73 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:49.129153 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac78ab80-504b-4961-8e18-b48ad6f54264 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:49.131610 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26282848-7a93-4cbe-a36e-f7dfdf9f34ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:49.132549 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26282848-7a93-4cbe-a36e-f7dfdf9f34ea None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:49.132985 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26282848-7a93-4cbe-a36e-f7dfdf9f34ea None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:49.133406 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26282848-7a93-4cbe-a36e-f7dfdf9f34ea None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:49.134340 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26282848-7a93-4cbe-a36e-f7dfdf9f34ea None admin] RBAC: Adding query filter params (name=network) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:49.134824 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26282848-7a93-4cbe-a36e-f7dfdf9f34ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:49.136402 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26282848-7a93-4cbe-a36e-f7dfdf9f34ea None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:49.139035 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:49.144032 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 396/772] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 21 15:51:48 2022] GET /identity/v3/services?name=network => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:49.145373 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26282848-7a93-4cbe-a36e-f7dfdf9f34ea None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:49.190945 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61bd43f-19ce-465c-8e7b-194a49e09ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:49.192008 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61bd43f-19ce-465c-8e7b-194a49e09ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:49.192486 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61bd43f-19ce-465c-8e7b-194a49e09ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:49.192945 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61bd43f-19ce-465c-8e7b-194a49e09ac5 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:49.194208 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61bd43f-19ce-465c-8e7b-194a49e09ac5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:49.196499 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61bd43f-19ce-465c-8e7b-194a49e09ac5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:49.206332 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 377/773] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jul 21 15:51:48 2022] GET /identity/v3/services => generated 772 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:49.207793 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61bd43f-19ce-465c-8e7b-194a49e09ac5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:49.215870 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:49.332020 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26282848-7a93-4cbe-a36e-f7dfdf9f34ea None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:49.415426 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc436b-bfdb-475a-873b-1dff04345241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:49.416187 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc436b-bfdb-475a-873b-1dff04345241 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:49.417979 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc436b-bfdb-475a-873b-1dff04345241 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:49.418431 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc436b-bfdb-475a-873b-1dff04345241 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:49.419341 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc436b-bfdb-475a-873b-1dff04345241 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:49.419842 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc436b-bfdb-475a-873b-1dff04345241 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:49.422950 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc436b-bfdb-475a-873b-1dff04345241 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:49.433410 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61bd43f-19ce-465c-8e7b-194a49e09ac5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:49.450729 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 397/774] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:49.456327 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc436b-bfdb-475a-873b-1dff04345241 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:49.482594 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a99bde6-1beb-46dd-a7a0-e00fecb000c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:49.484448 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99bde6-1beb-46dd-a7a0-e00fecb000c6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:49.484845 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99bde6-1beb-46dd-a7a0-e00fecb000c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:49.485260 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99bde6-1beb-46dd-a7a0-e00fecb000c6 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:49.486455 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a99bde6-1beb-46dd-a7a0-e00fecb000c6 None admin] RBAC: Adding query filter params (type=network) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:49.486883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a99bde6-1beb-46dd-a7a0-e00fecb000c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:49.488906 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a99bde6-1beb-46dd-a7a0-e00fecb000c6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:49.500344 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 378/775] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 21 15:51:49 2022] GET /identity/v3/services?type=network => generated 350 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:49.508829 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a99bde6-1beb-46dd-a7a0-e00fecb000c6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:49.545487 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:49.595676 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:49.599488 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc436b-bfdb-475a-873b-1dff04345241 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:49.645601 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f02cb-84d0-4ff7-8763-d110f5aa9c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:49.645601 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f02cb-84d0-4ff7-8763-d110f5aa9c11 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:49.645601 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f02cb-84d0-4ff7-8763-d110f5aa9c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:49.645601 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f02cb-84d0-4ff7-8763-d110f5aa9c11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:49.658662 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1f02cb-84d0-4ff7-8763-d110f5aa9c11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:49.663444 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1f02cb-84d0-4ff7-8763-d110f5aa9c11 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:49.679456 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a99bde6-1beb-46dd-a7a0-e00fecb000c6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:49.691900 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-8d1f02cb-84d0-4ff7-8763-d110f5aa9c11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:49.691900 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:49.691900 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:49.691900 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:49.691900 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:49.691900 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:49.691900 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:49.691900 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:49.691900 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:49.691900 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:49.691900 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:49.691900 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:49.691900 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:51:49.691900 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:51:49.691900 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:51:49.692891 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:51:49.692891 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:49.692891 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:49.692891 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:49.692891 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:49.692891 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:49.692891 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:49.692891 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:49.692891 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:49.692891 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:49.692891 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:49.692891 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:49.692891 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:49.692891 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:49.692891 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:49.692891 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:49.692891 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:49.693850 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:51:49.693850 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:51:49.693850 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:51:49.693850 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:51:49.693850 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:51:49.693850 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:51:49.693850 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:49.693850 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:49.705354 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 398/776] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:51:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:49.706361 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f02cb-84d0-4ff7-8763-d110f5aa9c11 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:49.747468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34aa7e-e499-4653-aec4-b3badf4aa0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:49.748171 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34aa7e-e499-4653-aec4-b3badf4aa0f3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:49.748460 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34aa7e-e499-4653-aec4-b3badf4aa0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:49.748740 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34aa7e-e499-4653-aec4-b3badf4aa0f3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:49.749680 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa34aa7e-e499-4653-aec4-b3badf4aa0f3 None admin] RBAC: Adding query filter params (interface=public, service_id=b90d5029e72d40298557fa49cbf82592, region_id=RegionOne) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:49.750003 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa34aa7e-e499-4653-aec4-b3badf4aa0f3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:49.751416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa34aa7e-e499-4653-aec4-b3badf4aa0f3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:49.761956 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 379/777] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Jul 21 15:51:49 2022] GET /identity/v3/endpoints?service_id=b90d5029e72d40298557fa49cbf82592&interface=public®ion_id=RegionOne => generated 194 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:49.763480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34aa7e-e499-4653-aec4-b3badf4aa0f3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:49.878246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34aa7e-e499-4653-aec4-b3badf4aa0f3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:49.913954 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:49.918206 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63578cf8-8b79-475a-a87d-61fd3f60f622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:49.918896 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63578cf8-8b79-475a-a87d-61fd3f60f622 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:49.919189 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63578cf8-8b79-475a-a87d-61fd3f60f622 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:49.919476 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63578cf8-8b79-475a-a87d-61fd3f60f622 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:49.920246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63578cf8-8b79-475a-a87d-61fd3f60f622 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:49.922060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63578cf8-8b79-475a-a87d-61fd3f60f622 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:49.928790 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 380/778] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jul 21 15:51:49 2022] GET /identity/v3/services => generated 772 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:50.000734 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f02cb-84d0-4ff7-8763-d110f5aa9c11 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:50.031528 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:50.052890 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82bf314-809d-40e0-affa-612940ff6503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:50.053592 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bf314-809d-40e0-affa-612940ff6503 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:50.053878 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bf314-809d-40e0-affa-612940ff6503 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:50.054155 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bf314-809d-40e0-affa-612940ff6503 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:50.055060 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82bf314-809d-40e0-affa-612940ff6503 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:50.055374 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82bf314-809d-40e0-affa-612940ff6503 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:50.056854 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82bf314-809d-40e0-affa-612940ff6503 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:50.074345 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 399/779] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:51:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:50.080612 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63578cf8-8b79-475a-a87d-61fd3f60f622 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:50.199256 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63578cf8-8b79-475a-a87d-61fd3f60f622 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:50.236892 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4035b-36e9-4c9f-9a3a-946ae72af7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:50.238006 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4035b-36e9-4c9f-9a3a-946ae72af7b1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:50.238518 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4035b-36e9-4c9f-9a3a-946ae72af7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:50.239058 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4035b-36e9-4c9f-9a3a-946ae72af7b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:50.245615 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4035b-36e9-4c9f-9a3a-946ae72af7b1 None admin] RBAC: Adding query filter params (role.id=64d94d0d5a7640818c21aa5a622624ab, scope.project.id=bb1ac00308714c39a79aae09fbb73021, user.id=6edbad98ed0f4823b9955f9f0e270149) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:50.245615 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4035b-36e9-4c9f-9a3a-946ae72af7b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:50.245615 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4035b-36e9-4c9f-9a3a-946ae72af7b1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:50.248400 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 381/780] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:51:50 2022] GET /identity/v3/role_assignments?user.id=6edbad98ed0f4823b9955f9f0e270149&scope.project.id=bb1ac00308714c39a79aae09fbb73021&role.id=64d94d0d5a7640818c21aa5a622624ab => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:50.970948 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76e3c9-19c3-4808-b84d-1660d297c149 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:50.971474 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76e3c9-19c3-4808-b84d-1660d297c149 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:50.971922 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76e3c9-19c3-4808-b84d-1660d297c149 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:50.973724 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 400/781] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:50.983196 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77b80b-c862-4e87-8fdc-9875fd5afca5 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:50.983696 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77b80b-c862-4e87-8fdc-9875fd5afca5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:50.984080 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77b80b-c862-4e87-8fdc-9875fd5afca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:51.006004 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6e77b80b-c862-4e87-8fdc-9875fd5afca5 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:51.007262 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6e77b80b-c862-4e87-8fdc-9875fd5afca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:51.079121 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:51.103623 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 382/782] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:51:51.137905 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f76e3c9-19c3-4808-b84d-1660d297c149 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:51.308482 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f76e3c9-19c3-4808-b84d-1660d297c149 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:51.361181 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c731250c-b6ea-4d2c-b6c1-d1f761d3ab06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:51.362028 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731250c-b6ea-4d2c-b6c1-d1f761d3ab06 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:51.362315 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731250c-b6ea-4d2c-b6c1-d1f761d3ab06 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:51.362592 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731250c-b6ea-4d2c-b6c1-d1f761d3ab06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:51.378851 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c731250c-b6ea-4d2c-b6c1-d1f761d3ab06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:51.380949 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96a843-7311-4965-af1e-8bdebb64553f None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:51.381249 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96a843-7311-4965-af1e-8bdebb64553f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:51.381532 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96a843-7311-4965-af1e-8bdebb64553f None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:51.386645 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 383/783] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:51.388135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-c731250c-b6ea-4d2c-b6c1-d1f761d3ab06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:51:51.388135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:51.388135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:51.388135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:51.388135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:51.388135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:51.388135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:51.388135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:51.388135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:51.388135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:51.388135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:51.388135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:51.388135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:51.388135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:51.388135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:51.388135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:51.389079 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:51.389079 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:51.389079 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:51.389079 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:51.389079 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:51.389079 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:51.389079 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:51.389079 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:51.389079 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:51.389079 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:51.389079 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:51.389079 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:51.389079 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:51.389079 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:51.389079 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:51.389079 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:51.390553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:51.390553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:51.390553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:51.390553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:51.390553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:51.390553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:51.390553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:51.390553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:51.390553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:51.390553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:51.390553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:51:51.390553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:51.391677 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ebd86-daaa-4ccb-b06d-bf6c27e1b0a5 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:51.391841 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ebd86-daaa-4ccb-b06d-bf6c27e1b0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:51.392107 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ebd86-daaa-4ccb-b06d-bf6c27e1b0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:51.393888 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 401/784] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:51:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:51.400116 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c731250c-b6ea-4d2c-b6c1-d1f761d3ab06 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:51.442721 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c94ebd86-daaa-4ccb-b06d-bf6c27e1b0a5 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:51.443956 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c94ebd86-daaa-4ccb-b06d-bf6c27e1b0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:51.446078 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:51.572790 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c731250c-b6ea-4d2c-b6c1-d1f761d3ab06 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:51.590538 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 384/785] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:51:51.615307 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94ebd86-daaa-4ccb-b06d-bf6c27e1b0a5 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:51.651376 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6418bc-aba2-4997-8bf9-10288f4858dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:51.652423 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6418bc-aba2-4997-8bf9-10288f4858dd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:51.653024 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6418bc-aba2-4997-8bf9-10288f4858dd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:51.653517 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6418bc-aba2-4997-8bf9-10288f4858dd None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:51.656311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6418bc-aba2-4997-8bf9-10288f4858dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:51.658039 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6418bc-aba2-4997-8bf9-10288f4858dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:51.659892 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6418bc-aba2-4997-8bf9-10288f4858dd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:51.677347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 402/786] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:51:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:51.685246 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6418bc-aba2-4997-8bf9-10288f4858dd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:51.813927 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94ebd86-daaa-4ccb-b06d-bf6c27e1b0a5 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:51.864116 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:51.875980 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6418bc-aba2-4997-8bf9-10288f4858dd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:51.877858 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:51.918648 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b66ba6-f523-4568-8e21-d546270b553a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:51.920133 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2b91f-0356-4fc4-9abb-a527f6784f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:51.921130 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b66ba6-f523-4568-8e21-d546270b553a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:51.922176 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2b91f-0356-4fc4-9abb-a527f6784f62 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:51.922622 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b66ba6-f523-4568-8e21-d546270b553a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:51.923059 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b66ba6-f523-4568-8e21-d546270b553a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:51.923480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2b91f-0356-4fc4-9abb-a527f6784f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:51.924498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2b91f-0356-4fc4-9abb-a527f6784f62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:51.946758 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b66ba6-f523-4568-8e21-d546270b553a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:51.948835 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b66ba6-f523-4568-8e21-d546270b553a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:51.950333 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef2b91f-0356-4fc4-9abb-a527f6784f62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:51.959175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-7ef2b91f-0356-4fc4-9abb-a527f6784f62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:51:51.959175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:51.959175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:51.959175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:51.959175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:51.959175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:51.959175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:51.959175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:51.959175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:51.959175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:51.959175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:51.959175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:51.959175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:51.959175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:51.959175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:51.959175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:51.960369 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:51.960369 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:51.960369 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:51.960369 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:51.960369 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:51.960369 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:51.960369 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:51.960369 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:51.960369 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:51.960369 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:51.960369 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:51.960369 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:51.960369 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:51.960369 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:51.960369 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:51.960369 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:51.961248 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:51.961248 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:51.961248 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:51.961248 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:51.961248 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:51.961248 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:51.961248 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:51.961248 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:51.961248 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:51.961248 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:51.961248 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:51:51.961248 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:51.964571 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 385/787] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:51:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:51:51.974258 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2b91f-0356-4fc4-9abb-a527f6784f62 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:51.993128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-80b66ba6-f523-4568-8e21-d546270b553a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:51.993128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:51.993128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:51.993128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:51.993128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:51.993128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:51.993128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:51.993128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:51.993128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:51.993128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:51.993128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:51.993128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:51.993128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:51.993128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:51.993128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:51.994291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:51.994291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:51.994291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:51.994291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:51.994291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:51.994291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:51.994291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:51.994291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:51.994291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:51.994291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:51.994291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:51.994291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:51.994291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:51.994291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:51.994291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:51.994291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:51.994291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:51.995235 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:51.995235 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:51.995235 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:51.995235 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:51.997173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 403/788] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:52.003369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b66ba6-f523-4568-8e21-d546270b553a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:52.117890 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b66ba6-f523-4568-8e21-d546270b553a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:52.158603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352afe66-36f0-4aa0-a30a-a63b8f7722b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:52.159975 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352afe66-36f0-4aa0-a30a-a63b8f7722b3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:52.160404 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352afe66-36f0-4aa0-a30a-a63b8f7722b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:52.160921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352afe66-36f0-4aa0-a30a-a63b8f7722b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:52.161852 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352afe66-36f0-4aa0-a30a-a63b8f7722b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:52.162308 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352afe66-36f0-4aa0-a30a-a63b8f7722b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:52.164144 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352afe66-36f0-4aa0-a30a-a63b8f7722b3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:52.174461 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2b91f-0356-4fc4-9abb-a527f6784f62 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:52.179867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 404/789] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:52.185777 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416d1a3-9b4c-4f46-ad69-c7395da64853 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:52.187132 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416d1a3-9b4c-4f46-ad69-c7395da64853 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:52.187566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416d1a3-9b4c-4f46-ad69-c7395da64853 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:52.189496 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 405/790] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:52 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:52.190473 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e416d1a3-9b4c-4f46-ad69-c7395da64853 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:52.213565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:52.225508 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c4709-e2bb-4e20-b104-b505e08a5b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:52.227069 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c4709-e2bb-4e20-b104-b505e08a5b1b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:52.227798 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c4709-e2bb-4e20-b104-b505e08a5b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:52.228277 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c4709-e2bb-4e20-b104-b505e08a5b1b None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:52.229922 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87c4709-e2bb-4e20-b104-b505e08a5b1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:52.230846 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87c4709-e2bb-4e20-b104-b505e08a5b1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:52.235773 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87c4709-e2bb-4e20-b104-b505e08a5b1b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:52.251597 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 386/791] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:51:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:51:52.253244 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d95fd-627a-4256-a720-dccec3f33ae6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:52.253784 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d95fd-627a-4256-a720-dccec3f33ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:52.254297 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d95fd-627a-4256-a720-dccec3f33ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:52.286916 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8a3d95fd-627a-4256-a720-dccec3f33ae6 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:52.289483 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8a3d95fd-627a-4256-a720-dccec3f33ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:52.333372 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e416d1a3-9b4c-4f46-ad69-c7395da64853 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:52.347504 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:52.384614 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c91f8-a9f2-405b-b26f-e9c7f638827d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:52.386541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c91f8-a9f2-405b-b26f-e9c7f638827d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:52.387317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c91f8-a9f2-405b-b26f-e9c7f638827d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:52.387824 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c91f8-a9f2-405b-b26f-e9c7f638827d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:52.398156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c91f8-a9f2-405b-b26f-e9c7f638827d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:52.399948 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c91f8-a9f2-405b-b26f-e9c7f638827d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:52.401717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 387/792] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:51:52.402710 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d95fd-627a-4256-a720-dccec3f33ae6 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:52.418751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-a38c91f8-a9f2-405b-b26f-e9c7f638827d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 15:51:52.418751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:52.418751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:52.418751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:52.418751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:52.418751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:52.418751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:52.418751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:52.418751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:52.418751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:52.418751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:52.418751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:52.418751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:51:52.418751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:51:52.418751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:51:52.418751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:51:52.419820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:52.419820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:52.419820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:51:52.419820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:52.419820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:51:52.419820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:52.419820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:52.419820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:52.419820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:52.419820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:52.419820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:52.419820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:52.419820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:52.419820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:52.419820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:52.419820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:52.419820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:52.419820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:52.420835 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:52.420835 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:52.420835 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:51:52.420835 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:51:52.420835 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:51:52.420835 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:51:52.420835 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:51:52.420835 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:51:52.420835 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 15:51:52.420835 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:52.424621 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 406/793] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:51:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:52.430705 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c91f8-a9f2-405b-b26f-e9c7f638827d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:52.537535 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:52.560372 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d95fd-627a-4256-a720-dccec3f33ae6 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:52.575341 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c91f8-a9f2-405b-b26f-e9c7f638827d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:52.610051 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7346a03-57f6-4d89-91cc-8000e97f2a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:52.611973 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7346a03-57f6-4d89-91cc-8000e97f2a26 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:52.612484 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7346a03-57f6-4d89-91cc-8000e97f2a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:52.613229 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7346a03-57f6-4d89-91cc-8000e97f2a26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:52.619072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98432e7b-5378-4b77-bb6c-24dd17682ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:52.619753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98432e7b-5378-4b77-bb6c-24dd17682ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:52.620038 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98432e7b-5378-4b77-bb6c-24dd17682ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:52.620315 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98432e7b-5378-4b77-bb6c-24dd17682ad5 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:52.621173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98432e7b-5378-4b77-bb6c-24dd17682ad5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:52.621486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98432e7b-5378-4b77-bb6c-24dd17682ad5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:52.622964 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98432e7b-5378-4b77-bb6c-24dd17682ad5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:52.629326 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7346a03-57f6-4d89-91cc-8000e97f2a26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:52.631504 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7346a03-57f6-4d89-91cc-8000e97f2a26 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:52.648677 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 407/794] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Jul 21 15:51:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:52.648677 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98432e7b-5378-4b77-bb6c-24dd17682ad5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:52.666198 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-e7346a03-57f6-4d89-91cc-8000e97f2a26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:52.666198 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:52.666198 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:52.666198 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:52.666198 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:52.666198 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:52.666198 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:52.666198 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:52.666198 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:52.666198 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:52.666198 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:52.666198 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:52.666198 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:52.666198 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:52.666198 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:52.667171 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:52.667171 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:52.667171 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:52.667171 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:52.667171 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:52.667171 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:52.667171 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:52.667171 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:52.667171 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:52.667171 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:52.667171 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:52.667171 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:52.667171 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:52.667171 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:52.667171 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:52.667171 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:52.667171 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:52.668089 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:52.668089 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:52.668089 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:52.668089 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:52.670579 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 388/795] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:52.671701 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7346a03-57f6-4d89-91cc-8000e97f2a26 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:52.704686 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:52.769677 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98432e7b-5378-4b77-bb6c-24dd17682ad5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:52.796318 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7346a03-57f6-4d89-91cc-8000e97f2a26 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:52.805188 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:52.812601 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22146ebf-a27f-4871-a2d3-8653d00f0924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:52.814040 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22146ebf-a27f-4871-a2d3-8653d00f0924 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:52.814326 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22146ebf-a27f-4871-a2d3-8653d00f0924 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:52.814754 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22146ebf-a27f-4871-a2d3-8653d00f0924 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:52.815439 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22146ebf-a27f-4871-a2d3-8653d00f0924 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:52.817264 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22146ebf-a27f-4871-a2d3-8653d00f0924 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:52.831576 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-22146ebf-a27f-4871-a2d3-8653d00f0924 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 15:51:52.831576 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:52.831576 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 15:51:52.831576 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 15:51:52.831576 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 15:51:52.831576 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 15:51:52.831576 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 15:51:52.831576 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:51:52.831576 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 15:51:52.831576 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:52.831576 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 15:51:52.831576 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:52.831576 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:52.831576 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:52.831576 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:52.831576 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:52.832605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:52.832605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:52.832605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:52.832605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:52.832605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:52.832605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:52.832605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:52.832605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 15:51:52.832605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 15:51:52.832605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 15:51:52.832605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 15:51:52.832605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:52.832605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:52.832605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:52.832605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:52.832605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:52.833475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:52.833475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:52.833475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:52.833475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:52.833475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:52.833475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:52.833475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:52.833475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:52.833475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:52.833475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:52.833475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:52.833475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 15:51:52.833475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:51:52.833475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 15:51:52.833475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:52.833475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 408/796] 10.222.0.52 () {56 vars in 1129 bytes} [Thu Jul 21 15:51:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:52.834397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22146ebf-a27f-4871-a2d3-8653d00f0924 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:52.847925 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c02b8fd-cad3-43cf-8a74-d7a554e1d61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:52.848801 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02b8fd-cad3-43cf-8a74-d7a554e1d61d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:52.849090 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02b8fd-cad3-43cf-8a74-d7a554e1d61d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:52.849372 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02b8fd-cad3-43cf-8a74-d7a554e1d61d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:52.862471 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c02b8fd-cad3-43cf-8a74-d7a554e1d61d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:52.863967 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c02b8fd-cad3-43cf-8a74-d7a554e1d61d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:52.884074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-0c02b8fd-cad3-43cf-8a74-d7a554e1d61d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:52.884074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:52.884074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:52.884074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:52.884074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:52.884074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:52.884074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:52.884074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:52.884074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:52.884074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:52.884074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:52.884074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:52.884074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:52.884074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:52.884074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:52.885241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:52.885241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:52.885241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:52.885241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:52.885241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:52.885241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:52.885241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:52.885241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:52.885241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:52.885241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:52.885241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:52.885241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:52.885241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:52.885241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:52.885241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:52.885241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:52.885241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:52.886473 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:52.886473 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:52.886473 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:52.886473 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:52.886473 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 389/797] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:52.886907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7e6d1-6267-4b84-b7cb-d2a42eee923d None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:52.887176 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7e6d1-6267-4b84-b7cb-d2a42eee923d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:52.887794 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7e6d1-6267-4b84-b7cb-d2a42eee923d None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:52.888733 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 390/798] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:52.889441 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa7e6d1-6267-4b84-b7cb-d2a42eee923d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:52.951437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22146ebf-a27f-4871-a2d3-8653d00f0924 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:52.970410 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:52.986737 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53429d95-181f-4f50-b311-6b297ca4660f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:52.987404 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53429d95-181f-4f50-b311-6b297ca4660f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:52.987682 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53429d95-181f-4f50-b311-6b297ca4660f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:52.987952 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53429d95-181f-4f50-b311-6b297ca4660f None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:52.988680 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53429d95-181f-4f50-b311-6b297ca4660f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:52.988987 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53429d95-181f-4f50-b311-6b297ca4660f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:52.990417 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53429d95-181f-4f50-b311-6b297ca4660f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:53.004486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 409/799] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:53.005866 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda527b-5beb-447e-801b-5dfa54042af8 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:53.006156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda527b-5beb-447e-801b-5dfa54042af8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:53.006425 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda527b-5beb-447e-801b-5dfa54042af8 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:53.007842 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 410/800] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:53.008356 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda527b-5beb-447e-801b-5dfa54042af8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:53.011193 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa7e6d1-6267-4b84-b7cb-d2a42eee923d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:53.056326 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1a281-4b1c-4578-8b13-f172cc22db45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:53.057010 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1a281-4b1c-4578-8b13-f172cc22db45 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:53.057280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1a281-4b1c-4578-8b13-f172cc22db45 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:53.057564 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1a281-4b1c-4578-8b13-f172cc22db45 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:53.058311 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a1a281-4b1c-4578-8b13-f172cc22db45 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:53.058623 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a1a281-4b1c-4578-8b13-f172cc22db45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:53.060026 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a1a281-4b1c-4578-8b13-f172cc22db45 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:53.064907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 391/801] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Jul 21 15:51:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:53.066132 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17867a98-f658-4a33-94c8-fa3204dcb04c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:53.066417 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17867a98-f658-4a33-94c8-fa3204dcb04c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:53.066685 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17867a98-f658-4a33-94c8-fa3204dcb04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:53.073112 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:53.083373 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-17867a98-f658-4a33-94c8-fa3204dcb04c None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:53.084601 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-17867a98-f658-4a33-94c8-fa3204dcb04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:53.122228 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda527b-5beb-447e-801b-5dfa54042af8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:53.159568 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b1778d-a92d-471a-b8a0-7f3a79bd3a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:53.160507 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1778d-a92d-471a-b8a0-7f3a79bd3a55 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:53.160906 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1778d-a92d-471a-b8a0-7f3a79bd3a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:53.161325 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1778d-a92d-471a-b8a0-7f3a79bd3a55 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:53.162420 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b1778d-a92d-471a-b8a0-7f3a79bd3a55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:53.162889 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b1778d-a92d-471a-b8a0-7f3a79bd3a55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:53.163418 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 392/802] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:51:53.164270 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17867a98-f658-4a33-94c8-fa3204dcb04c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:53.165012 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b1778d-a92d-471a-b8a0-7f3a79bd3a55 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:53.181647 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 411/803] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:53.182940 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65e0168-6c2c-4488-9dfd-4e9ed686a663 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:53.183260 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65e0168-6c2c-4488-9dfd-4e9ed686a663 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:53.183586 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65e0168-6c2c-4488-9dfd-4e9ed686a663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:53.203684 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d65e0168-6c2c-4488-9dfd-4e9ed686a663 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:53.205473 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d65e0168-6c2c-4488-9dfd-4e9ed686a663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:53.242400 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:53.284243 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17867a98-f658-4a33-94c8-fa3204dcb04c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:53.292548 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 412/804] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:51:53.293414 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65e0168-6c2c-4488-9dfd-4e9ed686a663 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:53.327161 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc6b17-1776-43f9-833e-c4ccb62b36bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:53.327962 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc6b17-1776-43f9-833e-c4ccb62b36bf None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:53.328250 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc6b17-1776-43f9-833e-c4ccb62b36bf None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:53.328523 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc6b17-1776-43f9-833e-c4ccb62b36bf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:53.336341 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc6b17-1776-43f9-833e-c4ccb62b36bf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:53.337858 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc6b17-1776-43f9-833e-c4ccb62b36bf None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:53.353753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-28bc6b17-1776-43f9-833e-c4ccb62b36bf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 15:51:53.353753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:53.353753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:53.353753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:53.353753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:53.353753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:53.353753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:53.353753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:53.353753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:53.353753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:53.353753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:53.353753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:53.353753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:51:53.353753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:51:53.353753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:51:53.353753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:51:53.354973 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:53.354973 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:53.354973 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:51:53.354973 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:53.354973 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:51:53.354973 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:53.354973 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:53.354973 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:53.354973 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:53.354973 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:53.354973 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:53.354973 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:53.354973 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:53.354973 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:53.354973 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:53.354973 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:53.354973 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:53.354973 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:53.355956 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:53.355956 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:53.355956 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:51:53.355956 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:51:53.355956 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:51:53.355956 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:51:53.355956 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:51:53.355956 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:51:53.355956 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 15:51:53.355956 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:53.355956 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 393/805] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jul 21 15:51:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:53.355956 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc6b17-1776-43f9-833e-c4ccb62b36bf None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:53.357539 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:53.421104 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65e0168-6c2c-4488-9dfd-4e9ed686a663 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:53.456348 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b99673-83f5-487e-87ff-b086c13dfde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:53.457014 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b99673-83f5-487e-87ff-b086c13dfde2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:53.457300 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b99673-83f5-487e-87ff-b086c13dfde2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:53.457583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b99673-83f5-487e-87ff-b086c13dfde2 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:53.458323 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b99673-83f5-487e-87ff-b086c13dfde2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:53.458641 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b99673-83f5-487e-87ff-b086c13dfde2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:53.460033 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b99673-83f5-487e-87ff-b086c13dfde2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:53.463397 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc6b17-1776-43f9-833e-c4ccb62b36bf None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:53.465055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 413/806] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Jul 21 15:51:53 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:53.466064 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b99673-83f5-487e-87ff-b086c13dfde2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:53.491582 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:53.500001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc0544-8d8c-4ce4-b3dc-40d9a2a0bb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:53.500802 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc0544-8d8c-4ce4-b3dc-40d9a2a0bb48 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:53.501100 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc0544-8d8c-4ce4-b3dc-40d9a2a0bb48 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:53.501383 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc0544-8d8c-4ce4-b3dc-40d9a2a0bb48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:53.508764 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bc0544-8d8c-4ce4-b3dc-40d9a2a0bb48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:53.510277 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bc0544-8d8c-4ce4-b3dc-40d9a2a0bb48 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:53.525188 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-75bc0544-8d8c-4ce4-b3dc-40d9a2a0bb48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:53.525188 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:53.525188 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:53.525188 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:53.525188 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:53.525188 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:53.525188 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:53.525188 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:53.525188 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:53.525188 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:53.525188 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:53.525188 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:53.525188 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:51:53.525188 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:51:53.525188 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:51:53.526272 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:51:53.526272 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:53.526272 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:53.526272 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:53.526272 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:53.526272 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:53.526272 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:53.526272 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:53.526272 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:53.526272 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:53.526272 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:53.526272 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:53.526272 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:53.526272 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:53.526272 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:53.526272 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:53.526272 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:53.527211 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:51:53.527211 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:51:53.527211 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:51:53.527211 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:51:53.527211 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:51:53.527211 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:51:53.527211 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:53.527211 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:53.527211 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 394/807] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:51:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:53.527712 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc0544-8d8c-4ce4-b3dc-40d9a2a0bb48 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:53.584511 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b99673-83f5-487e-87ff-b086c13dfde2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:53.596955 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:53.629219 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e099e-ad45-46dc-8a21-846fcab2f00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:53.631927 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e099e-ad45-46dc-8a21-846fcab2f00a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:53.631927 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e099e-ad45-46dc-8a21-846fcab2f00a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:53.631927 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e099e-ad45-46dc-8a21-846fcab2f00a None admin] PATH_INFO: `/v3/services/network` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:53.631927 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e099e-ad45-46dc-8a21-846fcab2f00a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:53.632634 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e099e-ad45-46dc-8a21-846fcab2f00a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:53.641704 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc0544-8d8c-4ce4-b3dc-40d9a2a0bb48 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:53.645415 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-f97e099e-ad45-46dc-8a21-846fcab2f00a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 15:51:53.645415 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:53.645415 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 15:51:53.645415 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 15:51:53.645415 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 15:51:53.645415 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 15:51:53.645415 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 15:51:53.645415 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:51:53.645415 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 15:51:53.645415 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:53.645415 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 15:51:53.645415 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:53.645415 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:53.645415 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:53.645415 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:53.645415 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:53.645415 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:53.646467 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:53.646467 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:53.646467 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:53.646467 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:53.646467 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:53.646467 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:53.646467 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 15:51:53.646467 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 15:51:53.646467 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 15:51:53.646467 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 15:51:53.646467 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:53.646467 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:53.646467 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:53.646467 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:53.646467 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:53.646467 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:53.647468 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:53.647468 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:53.647468 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:53.647468 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:53.647468 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:53.647468 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:53.647468 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:53.647468 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:53.647468 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:53.647468 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:53.647468 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 15:51:53.647468 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:51:53.647468 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 15:51:53.647468 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:53.647468 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 414/808] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:51:53 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:53.648276 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e099e-ad45-46dc-8a21-846fcab2f00a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:53.680006 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d5eb7-0dcf-4628-bfcd-167c6cb3a111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:53.680801 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d5eb7-0dcf-4628-bfcd-167c6cb3a111 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:53.681082 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d5eb7-0dcf-4628-bfcd-167c6cb3a111 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:53.681357 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d5eb7-0dcf-4628-bfcd-167c6cb3a111 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:53.694130 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d5eb7-0dcf-4628-bfcd-167c6cb3a111 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:53.702681 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-3f4d5eb7-0dcf-4628-bfcd-167c6cb3a111 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:53.702681 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:53.702681 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:53.702681 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:53.702681 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:53.702681 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:53.702681 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:53.702681 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:53.702681 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:53.702681 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:53.702681 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:53.702681 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:53.702681 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:53.702681 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:53.702681 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:53.702681 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:53.703803 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:53.703803 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:53.703803 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:53.703803 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:53.703803 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:53.703803 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:53.703803 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:53.703803 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:53.703803 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:53.703803 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:53.703803 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:53.703803 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:53.703803 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:53.703803 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:53.703803 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:53.703803 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:53.704768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:53.704768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:53.704768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:53.704768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:53.704768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:53.704768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:53.704768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:53.704768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:53.704768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:53.704768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:53.704768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:53.704768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:53.704768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 395/809] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 15:51:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:53.704768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d5eb7-0dcf-4628-bfcd-167c6cb3a111 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:53.761473 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:53.761969 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e099e-ad45-46dc-8a21-846fcab2f00a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:53.801169 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0256ee-5a9c-4895-81e6-87fbfe386a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:53.801874 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0256ee-5a9c-4895-81e6-87fbfe386a78 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:53.802160 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0256ee-5a9c-4895-81e6-87fbfe386a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:53.802439 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0256ee-5a9c-4895-81e6-87fbfe386a78 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:53.803296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0256ee-5a9c-4895-81e6-87fbfe386a78 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:53.803610 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0256ee-5a9c-4895-81e6-87fbfe386a78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:53.805093 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0256ee-5a9c-4895-81e6-87fbfe386a78 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:53.816672 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d5eb7-0dcf-4628-bfcd-167c6cb3a111 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:53.824643 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 415/810] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jul 21 15:51:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:53.825501 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0256ee-5a9c-4895-81e6-87fbfe386a78 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:53.852448 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a33b60-57e4-4d5d-876e-8674ace6ca5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:53.853112 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a33b60-57e4-4d5d-876e-8674ace6ca5c None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:53.853393 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a33b60-57e4-4d5d-876e-8674ace6ca5c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:53.853703 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a33b60-57e4-4d5d-876e-8674ace6ca5c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:53.854490 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a33b60-57e4-4d5d-876e-8674ace6ca5c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:53.855877 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a33b60-57e4-4d5d-876e-8674ace6ca5c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:53.869050 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:53.879382 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 396/811] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jul 21 15:51:53 2022] POST /identity/v3/endpoints => generated 346 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:51:53.881205 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a33b60-57e4-4d5d-876e-8674ace6ca5c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:53.959300 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0256ee-5a9c-4895-81e6-87fbfe386a78 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:54.015324 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff28c6-4aef-4e0f-b386-263d5e387848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:54.015809 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff28c6-4aef-4e0f-b386-263d5e387848 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:54.016235 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff28c6-4aef-4e0f-b386-263d5e387848 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:54.016503 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff28c6-4aef-4e0f-b386-263d5e387848 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:54.017392 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff28c6-4aef-4e0f-b386-263d5e387848 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:54.017799 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff28c6-4aef-4e0f-b386-263d5e387848 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:54.019482 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff28c6-4aef-4e0f-b386-263d5e387848 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:54.029564 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a33b60-57e4-4d5d-876e-8674ace6ca5c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:54.032701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 416/812] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:51:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:54.033606 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff28c6-4aef-4e0f-b386-263d5e387848 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:54.045670 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:54.083839 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c3d20-5b4c-4350-9771-6aa654802152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:54.084523 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c3d20-5b4c-4350-9771-6aa654802152 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:54.084812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c3d20-5b4c-4350-9771-6aa654802152 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:54.085089 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c3d20-5b4c-4350-9771-6aa654802152 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:54.086413 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1c3d20-5b4c-4350-9771-6aa654802152 None admin] RBAC: Adding query filter params (name=network) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:54.087371 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1c3d20-5b4c-4350-9771-6aa654802152 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:54.088151 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1c3d20-5b4c-4350-9771-6aa654802152 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:54.096658 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 397/813] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 21 15:51:53 2022] GET /identity/v3/services?name=network => generated 124 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:54.096658 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c3d20-5b4c-4350-9771-6aa654802152 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:54.141531 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:54.156046 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff28c6-4aef-4e0f-b386-263d5e387848 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:54.199029 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1330cb9d-792b-4670-adf8-fe612647f995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:54.199709 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330cb9d-792b-4670-adf8-fe612647f995 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:54.199999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330cb9d-792b-4670-adf8-fe612647f995 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:54.200279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330cb9d-792b-4670-adf8-fe612647f995 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:54.201049 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1330cb9d-792b-4670-adf8-fe612647f995 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:54.201379 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1330cb9d-792b-4670-adf8-fe612647f995 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:54.202832 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1330cb9d-792b-4670-adf8-fe612647f995 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:54.211873 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 417/814] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 15:51:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:54.212705 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1330cb9d-792b-4670-adf8-fe612647f995 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:54.220552 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c3d20-5b4c-4350-9771-6aa654802152 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:54.266817 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7555bb-c2c9-4f57-945a-db208661ae46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:54.268792 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7555bb-c2c9-4f57-945a-db208661ae46 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:54.269250 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7555bb-c2c9-4f57-945a-db208661ae46 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:54.269716 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7555bb-c2c9-4f57-945a-db208661ae46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:54.317283 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:54.339788 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1330cb9d-792b-4670-adf8-fe612647f995 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:54.360412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7555bb-c2c9-4f57-945a-db208661ae46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:54.361921 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7555bb-c2c9-4f57-945a-db208661ae46 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:54.380085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb21bb9f-a6d4-41e3-b01b-71ce52424744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:54.381257 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21bb9f-a6d4-41e3-b01b-71ce52424744 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:54.381759 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21bb9f-a6d4-41e3-b01b-71ce52424744 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:54.382222 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21bb9f-a6d4-41e3-b01b-71ce52424744 None admin] PATH_INFO: `/v3/projects/bb1ac00308714c39a79aae09fbb73021/users/f5bfae2cb30843b1b1a0343efef38638/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:54.384573 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb21bb9f-a6d4-41e3-b01b-71ce52424744 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb1ac00308714c39a79aae09fbb73021, user_id=f5bfae2cb30843b1b1a0343efef38638, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:54.386441 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb21bb9f-a6d4-41e3-b01b-71ce52424744 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:54.402602 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 418/815] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 15:51:54 2022] PUT /identity/v3/projects/bb1ac00308714c39a79aae09fbb73021/users/f5bfae2cb30843b1b1a0343efef38638/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 15:51:54.403498 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb21bb9f-a6d4-41e3-b01b-71ce52424744 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:54.429469 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:54.501532 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 398/816] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:51:54 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 15:51:54.503027 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7555bb-c2c9-4f57-945a-db208661ae46 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:54.545640 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb21bb9f-a6d4-41e3-b01b-71ce52424744 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:54.609960 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:54.618499 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94863b83-d828-40c5-9520-89b3ed8ad69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:54.619199 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94863b83-d828-40c5-9520-89b3ed8ad69e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:54.619489 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94863b83-d828-40c5-9520-89b3ed8ad69e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:54.619772 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94863b83-d828-40c5-9520-89b3ed8ad69e None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:54.620509 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94863b83-d828-40c5-9520-89b3ed8ad69e None admin] RBAC: Adding query filter params (type=network) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:54.620826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94863b83-d828-40c5-9520-89b3ed8ad69e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:54.622304 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94863b83-d828-40c5-9520-89b3ed8ad69e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:54.628283 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 419/817] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 21 15:51:54 2022] GET /identity/v3/services?type=network => generated 350 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:54.629033 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94863b83-d828-40c5-9520-89b3ed8ad69e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:54.652036 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7555bb-c2c9-4f57-945a-db208661ae46 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:54.692019 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e79818-262b-4bdf-918c-c39cad473c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:54.692823 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e79818-262b-4bdf-918c-c39cad473c8a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:54.693097 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e79818-262b-4bdf-918c-c39cad473c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:54.693373 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e79818-262b-4bdf-918c-c39cad473c8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:54.707197 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e79818-262b-4bdf-918c-c39cad473c8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:54.708669 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e79818-262b-4bdf-918c-c39cad473c8a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:54.725046 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:54.736928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-32e79818-262b-4bdf-918c-c39cad473c8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:54.736928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:54.736928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:54.736928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:54.736928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:54.736928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:54.736928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:54.736928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:54.736928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:54.736928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:54.736928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:54.736928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:54.736928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:54.736928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:54.736928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:54.738050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:54.738050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:54.738050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:54.738050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:54.738050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:54.738050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:54.738050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:54.738050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:54.738050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:54.738050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:54.738050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:54.738050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:54.738050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:54.738050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:54.738050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:54.738050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:54.738050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:54.738964 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:54.738964 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:54.738964 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:54.738964 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:54.741464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 399/818] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:54.742489 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e79818-262b-4bdf-918c-c39cad473c8a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:54.759727 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94863b83-d828-40c5-9520-89b3ed8ad69e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:54.810117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ebe2b4-1359-476b-b855-e91dcc140196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:54.811123 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebe2b4-1359-476b-b855-e91dcc140196 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:54.811537 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebe2b4-1359-476b-b855-e91dcc140196 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:54.811907 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebe2b4-1359-476b-b855-e91dcc140196 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:54.823821 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ebe2b4-1359-476b-b855-e91dcc140196 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:54.825526 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ebe2b4-1359-476b-b855-e91dcc140196 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:54.846154 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-c7ebe2b4-1359-476b-b855-e91dcc140196 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:54.846154 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:54.846154 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:54.846154 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:54.846154 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:54.846154 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:54.846154 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:54.846154 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:54.846154 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:54.846154 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:54.846154 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:54.846154 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:54.846154 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:54.846154 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:54.846154 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:54.847583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:54.847583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:54.847583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:54.847583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:54.847583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:54.847583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:54.847583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:54.847583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:54.847583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:54.847583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:54.847583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:54.847583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:54.847583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:54.847583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:54.847583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:54.847583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:54.847583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:54.848711 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:54.848711 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:54.848711 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:54.848711 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:54.854753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 420/819] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:54.855837 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ebe2b4-1359-476b-b855-e91dcc140196 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:54.902867 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e79818-262b-4bdf-918c-c39cad473c8a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:54.943574 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:54.968206 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faac01b-58e2-45e0-9da9-033a419bb7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:54.969054 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faac01b-58e2-45e0-9da9-033a419bb7f0 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:54.969441 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faac01b-58e2-45e0-9da9-033a419bb7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:54.970119 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faac01b-58e2-45e0-9da9-033a419bb7f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:54.970754 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faac01b-58e2-45e0-9da9-033a419bb7f0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:54.972569 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faac01b-58e2-45e0-9da9-033a419bb7f0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:55.005610 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 400/820] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jul 21 15:51:54 2022] POST /identity/v3/endpoints => generated 327 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:51:55.005610 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faac01b-58e2-45e0-9da9-033a419bb7f0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:55.018205 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ebe2b4-1359-476b-b855-e91dcc140196 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:55.050072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:55.075343 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae013d-2b4a-4d3d-ac7e-b55829ecd012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:55.075947 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae013d-2b4a-4d3d-ac7e-b55829ecd012 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:55.076227 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae013d-2b4a-4d3d-ac7e-b55829ecd012 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:55.076490 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae013d-2b4a-4d3d-ac7e-b55829ecd012 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:55.077825 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bae013d-2b4a-4d3d-ac7e-b55829ecd012 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:55.078136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bae013d-2b4a-4d3d-ac7e-b55829ecd012 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:55.082578 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bae013d-2b4a-4d3d-ac7e-b55829ecd012 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:55.112966 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 421/821] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:55.126856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae013d-2b4a-4d3d-ac7e-b55829ecd012 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:55.192153 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faac01b-58e2-45e0-9da9-033a419bb7f0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:55.250613 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa09d5d-6d32-4c76-9429-c4042ea3ce13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:55.251294 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa09d5d-6d32-4c76-9429-c4042ea3ce13 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:55.251575 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa09d5d-6d32-4c76-9429-c4042ea3ce13 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:55.251854 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa09d5d-6d32-4c76-9429-c4042ea3ce13 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:55.252593 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa09d5d-6d32-4c76-9429-c4042ea3ce13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:55.252901 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa09d5d-6d32-4c76-9429-c4042ea3ce13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:55.254341 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa09d5d-6d32-4c76-9429-c4042ea3ce13 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:55.269043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 401/822] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:55.274814 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa09d5d-6d32-4c76-9429-c4042ea3ce13 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:55.319601 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae013d-2b4a-4d3d-ac7e-b55829ecd012 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:55.328474 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:55.388042 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09aaa67-65b5-4b91-a3ee-e6a7a4655f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:55.388607 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09aaa67-65b5-4b91-a3ee-e6a7a4655f03 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:55.388895 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09aaa67-65b5-4b91-a3ee-e6a7a4655f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:55.389163 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09aaa67-65b5-4b91-a3ee-e6a7a4655f03 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:55.397810 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09aaa67-65b5-4b91-a3ee-e6a7a4655f03 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:55.399460 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09aaa67-65b5-4b91-a3ee-e6a7a4655f03 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:55.419746 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-a09aaa67-65b5-4b91-a3ee-e6a7a4655f03 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 15:51:55.419746 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:55.419746 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:55.419746 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:55.419746 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:55.419746 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:55.419746 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:55.419746 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:55.419746 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:55.419746 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:55.419746 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:55.419746 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:55.419746 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:51:55.419746 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:51:55.419746 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:51:55.419746 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:51:55.421548 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:55.421548 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:55.421548 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:51:55.421548 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:55.421548 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:51:55.421548 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:55.421548 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:55.421548 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:55.421548 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:55.421548 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:55.421548 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:55.421548 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:55.421548 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:55.421548 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:55.421548 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:55.421548 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:55.421548 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:55.421548 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:55.422644 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:55.422644 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:55.422644 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:51:55.422644 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:51:55.422644 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:51:55.422644 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:51:55.422644 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:51:55.422644 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:51:55.422644 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 15:51:55.422644 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:55.426986 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 422/823] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jul 21 15:51:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:55.428266 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09aaa67-65b5-4b91-a3ee-e6a7a4655f03 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:55.430251 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa09d5d-6d32-4c76-9429-c4042ea3ce13 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:55.476879 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b301357a-48ad-43b1-a956-3e1aaa5afc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:55.478069 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301357a-48ad-43b1-a956-3e1aaa5afc4d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:55.480488 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301357a-48ad-43b1-a956-3e1aaa5afc4d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:55.480996 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301357a-48ad-43b1-a956-3e1aaa5afc4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:55.483538 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:55.489298 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b301357a-48ad-43b1-a956-3e1aaa5afc4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:55.491051 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b301357a-48ad-43b1-a956-3e1aaa5afc4d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:55.512621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b301357a-48ad-43b1-a956-3e1aaa5afc4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:55.512621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:55.512621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:55.512621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:55.512621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:55.512621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:55.512621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:55.512621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:55.512621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:55.512621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:55.512621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:55.512621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:55.512621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:51:55.512621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:51:55.512621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:51:55.513812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:51:55.513812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:55.513812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:55.513812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:55.513812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:55.513812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:55.513812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:55.513812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:55.513812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:55.513812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:55.513812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:55.513812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:55.513812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:55.513812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:55.513812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:55.513812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:55.513812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:55.515068 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:51:55.515068 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:51:55.515068 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:51:55.515068 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:51:55.515068 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:51:55.515068 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:51:55.515068 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:55.515068 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:55.516839 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 402/824] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:51:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:55.522528 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b301357a-48ad-43b1-a956-3e1aaa5afc4d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:55.554219 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09aaa67-65b5-4b91-a3ee-e6a7a4655f03 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:55.594641 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4fbcd-d051-4d90-b29a-93f37cb1ed25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:55.595625 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4fbcd-d051-4d90-b29a-93f37cb1ed25 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:55.596053 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4fbcd-d051-4d90-b29a-93f37cb1ed25 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:55.596473 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4fbcd-d051-4d90-b29a-93f37cb1ed25 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:55.597761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e4fbcd-d051-4d90-b29a-93f37cb1ed25 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:55.598324 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e4fbcd-d051-4d90-b29a-93f37cb1ed25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:55.600115 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e4fbcd-d051-4d90-b29a-93f37cb1ed25 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:55.620093 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 423/825] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Jul 21 15:51:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:55.628072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4fbcd-d051-4d90-b29a-93f37cb1ed25 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:55.630088 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:55.641165 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b301357a-48ad-43b1-a956-3e1aaa5afc4d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:55.688521 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31813436-889f-4819-9cd1-912f5f51ce8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:55.689560 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31813436-889f-4819-9cd1-912f5f51ce8e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:55.690654 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31813436-889f-4819-9cd1-912f5f51ce8e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:55.691124 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31813436-889f-4819-9cd1-912f5f51ce8e None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:55.692175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31813436-889f-4819-9cd1-912f5f51ce8e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:55.693222 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31813436-889f-4819-9cd1-912f5f51ce8e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:55.695377 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31813436-889f-4819-9cd1-912f5f51ce8e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:55.712908 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 403/826] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:51:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:51:55.722805 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31813436-889f-4819-9cd1-912f5f51ce8e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:55.780372 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:55.784163 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4fbcd-d051-4d90-b29a-93f37cb1ed25 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:55.823493 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7051ec21-ddf7-4531-a09b-3eabcc765d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:55.824880 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051ec21-ddf7-4531-a09b-3eabcc765d5f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:55.825408 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051ec21-ddf7-4531-a09b-3eabcc765d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:55.825950 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051ec21-ddf7-4531-a09b-3eabcc765d5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:55.844640 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7051ec21-ddf7-4531-a09b-3eabcc765d5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:55.847605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7051ec21-ddf7-4531-a09b-3eabcc765d5f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:55.869242 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-7051ec21-ddf7-4531-a09b-3eabcc765d5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:55.869242 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:55.869242 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:55.869242 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:55.869242 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:55.869242 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:55.869242 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:55.869242 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:55.869242 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:55.869242 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:55.869242 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:55.869242 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:55.869242 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:55.869242 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:55.869242 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:55.870773 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:55.870773 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:55.870773 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:55.870773 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:55.870773 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:55.870773 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:55.870773 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:55.870773 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:55.870773 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:55.870773 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:55.870773 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:55.870773 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:55.870773 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:55.870773 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:55.870773 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:55.870773 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:55.870773 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:55.872221 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:55.872221 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:55.872221 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:55.872221 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:55.875354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31813436-889f-4819-9cd1-912f5f51ce8e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:55.875755 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 424/827] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:55.877060 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7051ec21-ddf7-4531-a09b-3eabcc765d5f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:55.919718 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46863868-8e6b-4a0f-a148-df40ead1fbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:55.921291 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46863868-8e6b-4a0f-a148-df40ead1fbf0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:55.921849 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46863868-8e6b-4a0f-a148-df40ead1fbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:55.922393 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46863868-8e6b-4a0f-a148-df40ead1fbf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:55.923499 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46863868-8e6b-4a0f-a148-df40ead1fbf0 None admin] RBAC: Adding query filter params (role.id=475521b8a0174037b162f92022aea1a1, scope.project.id=bb1ac00308714c39a79aae09fbb73021, user.id=092553a86f8848dba8c7271f5c0b2c46) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:55.924080 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46863868-8e6b-4a0f-a148-df40ead1fbf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:55.925776 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46863868-8e6b-4a0f-a148-df40ead1fbf0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:55.937956 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 404/828] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:51:55 2022] GET /identity/v3/role_assignments?user.id=092553a86f8848dba8c7271f5c0b2c46&scope.project.id=bb1ac00308714c39a79aae09fbb73021&role.id=475521b8a0174037b162f92022aea1a1 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:55.989285 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7051ec21-ddf7-4531-a09b-3eabcc765d5f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:56.029739 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b868c8-2139-4f2a-bc81-1884d8e2f1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:56.031299 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b868c8-2139-4f2a-bc81-1884d8e2f1f7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:56.031581 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b868c8-2139-4f2a-bc81-1884d8e2f1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:56.031856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b868c8-2139-4f2a-bc81-1884d8e2f1f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:56.032783 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b868c8-2139-4f2a-bc81-1884d8e2f1f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:56.033091 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b868c8-2139-4f2a-bc81-1884d8e2f1f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:56.034542 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b868c8-2139-4f2a-bc81-1884d8e2f1f7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:56.048152 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 425/829] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:51:56.053801 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46863868-8e6b-4a0f-a148-df40ead1fbf0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:56.079635 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:56.147681 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4c53b-4863-41f6-8691-8b59418f639b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:56.147979 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4c53b-4863-41f6-8691-8b59418f639b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:56.148159 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4c53b-4863-41f6-8691-8b59418f639b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:56.150419 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 426/830] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 15:51:56.157511 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463df3f-9d4a-4093-903f-89739cb0f174 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:56.157838 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463df3f-9d4a-4093-903f-89739cb0f174 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:56.158119 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463df3f-9d4a-4093-903f-89739cb0f174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:56.179000 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9463df3f-9d4a-4093-903f-89739cb0f174 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:56.180698 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9463df3f-9d4a-4093-903f-89739cb0f174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:56.182884 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46863868-8e6b-4a0f-a148-df40ead1fbf0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:56.192288 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:56.256737 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25170efc-5bf6-49b5-8a8a-896ac21a2318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:56.258244 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25170efc-5bf6-49b5-8a8a-896ac21a2318 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:56.259234 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25170efc-5bf6-49b5-8a8a-896ac21a2318 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:56.259805 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25170efc-5bf6-49b5-8a8a-896ac21a2318 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:56.270349 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25170efc-5bf6-49b5-8a8a-896ac21a2318 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:56.273511 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25170efc-5bf6-49b5-8a8a-896ac21a2318 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:56.277505 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 427/831] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 15:51:56.289997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-25170efc-5bf6-49b5-8a8a-896ac21a2318 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:56.289997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:56.289997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:56.289997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:56.289997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:56.289997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:56.289997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:56.289997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:56.289997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:56.289997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:56.289997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:56.289997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:56.289997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:51:56.289997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:51:56.289997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:51:56.290959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:51:56.290959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:56.290959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:56.290959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:56.290959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:56.290959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:56.290959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:56.290959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:56.290959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:56.290959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:56.290959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:56.290959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:56.290959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:56.290959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:56.290959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:56.290959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:56.290959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:56.291883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:51:56.291883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:51:56.291883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:51:56.291883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:51:56.291883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:51:56.291883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:51:56.291883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:56.291883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:56.294389 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 405/832] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:51:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:56.301127 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9463df3f-9d4a-4093-903f-89739cb0f174 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:56.302757 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25170efc-5bf6-49b5-8a8a-896ac21a2318 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:56.417991 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9463df3f-9d4a-4093-903f-89739cb0f174 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:56.423227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:56.463238 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25170efc-5bf6-49b5-8a8a-896ac21a2318 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:56.470921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485b6d7a-393b-49c6-a9c4-26d171ea5a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:56.473848 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b6d7a-393b-49c6-a9c4-26d171ea5a9a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:56.475195 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b6d7a-393b-49c6-a9c4-26d171ea5a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:56.475999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b6d7a-393b-49c6-a9c4-26d171ea5a9a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:56.478491 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485b6d7a-393b-49c6-a9c4-26d171ea5a9a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:56.482605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485b6d7a-393b-49c6-a9c4-26d171ea5a9a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:56.506528 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-485b6d7a-393b-49c6-a9c4-26d171ea5a9a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 15:51:56.506528 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:56.506528 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 15:51:56.506528 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 15:51:56.506528 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 15:51:56.506528 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 15:51:56.506528 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 15:51:56.506528 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:51:56.506528 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 15:51:56.506528 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:56.506528 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 15:51:56.506528 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:56.506528 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:56.506528 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:56.506528 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:56.506528 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:56.506528 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:56.508527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:56.508527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:56.508527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:56.508527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:56.508527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:56.508527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:56.508527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 15:51:56.508527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 15:51:56.508527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 15:51:56.508527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 15:51:56.508527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:56.508527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:56.508527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:56.508527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:56.508527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:56.508527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:56.510317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:56.510317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:56.510317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:56.510317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:56.510317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:56.510317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:56.510317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:56.510317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:56.510317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:56.510317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:56.510317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 15:51:56.510317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:51:56.510317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 15:51:56.510317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:56.512981 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 428/833] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 15:51:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:56.513930 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485b6d7a-393b-49c6-a9c4-26d171ea5a9a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:56.516652 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af101381-538c-4754-a65b-9438eaa4843c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:56.519954 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af101381-538c-4754-a65b-9438eaa4843c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:56.520746 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af101381-538c-4754-a65b-9438eaa4843c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:56.521252 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af101381-538c-4754-a65b-9438eaa4843c None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:56.522332 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af101381-538c-4754-a65b-9438eaa4843c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:56.522877 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af101381-538c-4754-a65b-9438eaa4843c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:56.525736 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:56.525930 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af101381-538c-4754-a65b-9438eaa4843c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:56.562214 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 406/834] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:51:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:51:56.566205 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af101381-538c-4754-a65b-9438eaa4843c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:56.652219 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485b6d7a-393b-49c6-a9c4-26d171ea5a9a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:56.704894 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b9644-64ae-477e-bd37-88f1f38aeb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:56.705739 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af101381-538c-4754-a65b-9438eaa4843c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:56.706374 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b9644-64ae-477e-bd37-88f1f38aeb25 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:56.707186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b9644-64ae-477e-bd37-88f1f38aeb25 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:56.708105 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b9644-64ae-477e-bd37-88f1f38aeb25 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:56.709044 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7b9644-64ae-477e-bd37-88f1f38aeb25 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:56.710289 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7b9644-64ae-477e-bd37-88f1f38aeb25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:56.712120 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7b9644-64ae-477e-bd37-88f1f38aeb25 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:56.726164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 429/835] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 15:51:56 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:56.728624 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b9644-64ae-477e-bd37-88f1f38aeb25 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:56.749438 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55100526-6c85-42aa-9d58-19aa48345826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:56.750770 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55100526-6c85-42aa-9d58-19aa48345826 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:56.751134 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55100526-6c85-42aa-9d58-19aa48345826 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:56.751714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55100526-6c85-42aa-9d58-19aa48345826 None admin] PATH_INFO: `/v3/projects/bb1ac00308714c39a79aae09fbb73021/users/6edbad98ed0f4823b9955f9f0e270149/roles/64d94d0d5a7640818c21aa5a622624ab` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:56.765874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:56.771920 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55100526-6c85-42aa-9d58-19aa48345826 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb1ac00308714c39a79aae09fbb73021, user_id=6edbad98ed0f4823b9955f9f0e270149, role_id=64d94d0d5a7640818c21aa5a622624ab)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:56.774108 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55100526-6c85-42aa-9d58-19aa48345826 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:56.796280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 407/836] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 15:51:56 2022] PUT /identity/v3/projects/bb1ac00308714c39a79aae09fbb73021/users/6edbad98ed0f4823b9955f9f0e270149/roles/64d94d0d5a7640818c21aa5a622624ab => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 15:51:56.798986 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071d542-71e8-4952-9ef5-1caaec968a0a None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:56.799587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071d542-71e8-4952-9ef5-1caaec968a0a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:56.800207 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071d542-71e8-4952-9ef5-1caaec968a0a None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:56.803235 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 408/837] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:56.808572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1610a7-6911-439f-ad2c-b42fad0f9617 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:56.808855 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1610a7-6911-439f-ad2c-b42fad0f9617 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:56.809134 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1610a7-6911-439f-ad2c-b42fad0f9617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:56.826232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:56.829351 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8b1610a7-6911-439f-ad2c-b42fad0f9617 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:56.830939 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8b1610a7-6911-439f-ad2c-b42fad0f9617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:56.868353 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b9644-64ae-477e-bd37-88f1f38aeb25 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:56.915321 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c545e26c-fc2e-4589-8828-dff778c22b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:56.916011 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545e26c-fc2e-4589-8828-dff778c22b36 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:56.916404 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545e26c-fc2e-4589-8828-dff778c22b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:56.916691 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545e26c-fc2e-4589-8828-dff778c22b36 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:56.917538 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c545e26c-fc2e-4589-8828-dff778c22b36 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:56.917882 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c545e26c-fc2e-4589-8828-dff778c22b36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:56.919314 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c545e26c-fc2e-4589-8828-dff778c22b36 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:56.922513 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 409/838] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 15:51:56.933755 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 430/839] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 15:51:56 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:51:56.948675 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1610a7-6911-439f-ad2c-b42fad0f9617 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:57.092338 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1610a7-6911-439f-ad2c-b42fad0f9617 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:57.118985 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:57.133566 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b801a01-a582-46c0-856f-d16780149cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:57.135143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b801a01-a582-46c0-856f-d16780149cbf None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:57.135746 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b801a01-a582-46c0-856f-d16780149cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:57.136253 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b801a01-a582-46c0-856f-d16780149cbf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:57.149100 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b801a01-a582-46c0-856f-d16780149cbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:57.157338 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-4b801a01-a582-46c0-856f-d16780149cbf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:51:57.157338 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:57.157338 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:57.157338 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:57.157338 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:57.157338 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:57.157338 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:57.157338 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:57.157338 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:57.157338 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:57.157338 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:57.157338 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:57.157338 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:57.157338 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:57.157338 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:57.157338 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:57.158387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:57.158387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:57.158387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:57.158387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:57.158387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:57.158387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:57.158387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:57.158387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:57.158387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:57.158387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:57.158387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:57.158387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:57.158387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:57.158387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:57.158387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:57.158387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:57.159246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:57.159246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:57.159246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:57.159246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:57.159246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:57.159246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:57.159246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:57.159246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:57.159246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:57.159246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:57.159246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:51:57.159246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:57.161664 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 410/840] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:51:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:57.167808 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c545e26c-fc2e-4589-8828-dff778c22b36 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:57.288657 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c545e26c-fc2e-4589-8828-dff778c22b36 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:57.330752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d180e507-e62d-44f9-ab83-c70719782fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:57.331430 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180e507-e62d-44f9-ab83-c70719782fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:57.331718 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180e507-e62d-44f9-ab83-c70719782fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:57.332000 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180e507-e62d-44f9-ab83-c70719782fe0 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:57.332769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d180e507-e62d-44f9-ab83-c70719782fe0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:57.333081 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d180e507-e62d-44f9-ab83-c70719782fe0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:57.334500 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d180e507-e62d-44f9-ab83-c70719782fe0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:57.335990 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:57.347937 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 431/841] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:51:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:57.355023 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b801a01-a582-46c0-856f-d16780149cbf None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:57.477484 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b801a01-a582-46c0-856f-d16780149cbf None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:57.514505 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3de8fd-d621-463f-b3af-19d41a25cf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:57.515367 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3de8fd-d621-463f-b3af-19d41a25cf79 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:57.515657 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3de8fd-d621-463f-b3af-19d41a25cf79 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:57.515938 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3de8fd-d621-463f-b3af-19d41a25cf79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:57.527630 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3de8fd-d621-463f-b3af-19d41a25cf79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:57.529111 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3de8fd-d621-463f-b3af-19d41a25cf79 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:57.547671 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-3d3de8fd-d621-463f-b3af-19d41a25cf79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:57.547671 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:57.547671 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:57.547671 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:57.547671 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:57.547671 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:57.547671 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:57.547671 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:57.547671 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:57.547671 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:57.547671 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:57.547671 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:57.547671 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:57.547671 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:57.547671 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:57.548716 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:57.548716 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:57.548716 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:57.548716 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:57.548716 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:57.548716 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:57.548716 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:57.548716 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:57.548716 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:57.548716 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:57.548716 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:57.548716 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:57.548716 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:57.548716 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:57.548716 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:57.548716 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:57.548716 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:57.549677 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:57.549677 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:57.549677 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:57.549677 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:57.549677 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 411/842] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:57.556182 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d180e507-e62d-44f9-ab83-c70719782fe0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:57.797458 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d180e507-e62d-44f9-ab83-c70719782fe0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:57.834601 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21f7a2-9324-4a69-b505-a78bf3f10114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:57.835278 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21f7a2-9324-4a69-b505-a78bf3f10114 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:57.835576 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21f7a2-9324-4a69-b505-a78bf3f10114 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:57.835834 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21f7a2-9324-4a69-b505-a78bf3f10114 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:57.836571 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba21f7a2-9324-4a69-b505-a78bf3f10114 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:57.836879 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba21f7a2-9324-4a69-b505-a78bf3f10114 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:57.838307 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba21f7a2-9324-4a69-b505-a78bf3f10114 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:57.851784 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 432/843] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:57.859295 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3de8fd-d621-463f-b3af-19d41a25cf79 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:57.898048 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:57.981116 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3de8fd-d621-463f-b3af-19d41a25cf79 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:58.018108 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca8500-e347-42a5-bbdf-f84cbb9dd4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:58.019335 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca8500-e347-42a5-bbdf-f84cbb9dd4b7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:58.019821 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca8500-e347-42a5-bbdf-f84cbb9dd4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:58.020306 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca8500-e347-42a5-bbdf-f84cbb9dd4b7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:58.030191 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ca8500-e347-42a5-bbdf-f84cbb9dd4b7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:58.032025 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ca8500-e347-42a5-bbdf-f84cbb9dd4b7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:58.048200 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-11ca8500-e347-42a5-bbdf-f84cbb9dd4b7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 15:51:58.048200 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:58.048200 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:58.048200 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:58.048200 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:58.048200 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:58.048200 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:58.048200 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:58.048200 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:58.048200 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:58.048200 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:58.048200 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:58.048200 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:51:58.048200 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:51:58.048200 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:51:58.048200 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:51:58.049224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:58.049224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:58.049224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:51:58.049224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:58.049224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:51:58.049224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:51:58.049224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:58.049224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:58.049224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:58.049224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:58.049224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:58.049224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:58.049224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:58.049224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:58.049224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:58.049224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:58.049224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:58.049224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:58.050259 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:58.050259 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:58.050259 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:51:58.050259 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:51:58.050259 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:51:58.050259 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:51:58.050259 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:51:58.050259 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:51:58.050259 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 15:51:58.050259 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:58.052901 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 412/844] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:51:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:58.058721 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21f7a2-9324-4a69-b505-a78bf3f10114 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:58.131917 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:58.197619 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21f7a2-9324-4a69-b505-a78bf3f10114 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:58.257603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea438875-5b2c-4b32-a1db-faaab6edfc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:58.257603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea438875-5b2c-4b32-a1db-faaab6edfc65 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:58.257603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea438875-5b2c-4b32-a1db-faaab6edfc65 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:58.257603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea438875-5b2c-4b32-a1db-faaab6edfc65 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:58.258055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea438875-5b2c-4b32-a1db-faaab6edfc65 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:58.258055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea438875-5b2c-4b32-a1db-faaab6edfc65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:58.258055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea438875-5b2c-4b32-a1db-faaab6edfc65 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:58.275189 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e646f-3aad-41fe-be3c-157eaae37773 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:58.275544 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e646f-3aad-41fe-be3c-157eaae37773 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:58.275745 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e646f-3aad-41fe-be3c-157eaae37773 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:58.276790 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 433/845] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Jul 21 15:51:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:58.277085 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 413/846] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:58.282691 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ed0c2-24a3-479d-abd9-b8468b8c1995 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:58.282844 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832e646f-3aad-41fe-be3c-157eaae37773 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:58.282999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ed0c2-24a3-479d-abd9-b8468b8c1995 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:58.283422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ed0c2-24a3-479d-abd9-b8468b8c1995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:58.300881 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2e6ed0c2-24a3-479d-abd9-b8468b8c1995 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:58.302136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2e6ed0c2-24a3-479d-abd9-b8468b8c1995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:58.386139 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:58.396454 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832e646f-3aad-41fe-be3c-157eaae37773 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:58.401745 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 434/847] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:51:58.430780 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ed0c2-24a3-479d-abd9-b8468b8c1995 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:58.456380 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f244b1-9a14-4a92-bad4-6c9480e2e0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:58.457048 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f244b1-9a14-4a92-bad4-6c9480e2e0df None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:58.457332 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f244b1-9a14-4a92-bad4-6c9480e2e0df None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:58.457646 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f244b1-9a14-4a92-bad4-6c9480e2e0df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:58.492492 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:58.556470 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ed0c2-24a3-479d-abd9-b8468b8c1995 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:58.560679 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f244b1-9a14-4a92-bad4-6c9480e2e0df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:58.563008 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f244b1-9a14-4a92-bad4-6c9480e2e0df None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:58.596495 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbb9ae-e589-4f2f-938c-14982e82d5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:58.597565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbb9ae-e589-4f2f-938c-14982e82d5aa None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:58.598086 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbb9ae-e589-4f2f-938c-14982e82d5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:58.598378 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbb9ae-e589-4f2f-938c-14982e82d5aa None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:58.599089 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bbb9ae-e589-4f2f-938c-14982e82d5aa None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:58.601168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bbb9ae-e589-4f2f-938c-14982e82d5aa None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:58.617917 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-06bbb9ae-e589-4f2f-938c-14982e82d5aa None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 15:51:58.617917 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:58.617917 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 15:51:58.617917 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 15:51:58.617917 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 15:51:58.617917 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 15:51:58.617917 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 15:51:58.617917 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:51:58.617917 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 15:51:58.617917 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:58.617917 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 15:51:58.617917 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:58.617917 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:58.617917 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:58.617917 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:58.617917 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:58.618999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:58.618999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:58.618999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:58.618999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:58.618999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:58.618999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:58.618999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:58.618999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 15:51:58.618999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 15:51:58.618999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 15:51:58.618999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 15:51:58.618999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:58.618999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:58.618999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:58.618999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:58.618999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:58.619906 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:58.619906 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:58.619906 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:58.619906 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:58.619906 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:58.619906 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:58.619906 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:58.619906 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:58.619906 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:58.619906 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:58.619906 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:58.619906 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 15:51:58.619906 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:51:58.619906 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 15:51:58.619906 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:58.624734 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 435/848] 10.222.0.52 () {56 vars in 1132 bytes} [Thu Jul 21 15:51:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:51:58.625717 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbb9ae-e589-4f2f-938c-14982e82d5aa None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:58.701090 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:58.718138 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 414/849] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:51:58 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 15:51:58.723220 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f244b1-9a14-4a92-bad4-6c9480e2e0df None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:58.769301 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbb9ae-e589-4f2f-938c-14982e82d5aa None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:58.781139 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:58.810657 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf5235-8976-47b0-96b2-358ab9568bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:58.811782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf5235-8976-47b0-96b2-358ab9568bcc None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:58.812264 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf5235-8976-47b0-96b2-358ab9568bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:58.812744 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf5235-8976-47b0-96b2-358ab9568bcc None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:58.813767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf5235-8976-47b0-96b2-358ab9568bcc None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:58.814381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf5235-8976-47b0-96b2-358ab9568bcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:58.816673 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf5235-8976-47b0-96b2-358ab9568bcc None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:58.822507 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 436/850] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jul 21 15:51:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:58.828950 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf5235-8976-47b0-96b2-358ab9568bcc None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:58.837591 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f244b1-9a14-4a92-bad4-6c9480e2e0df None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:58.875695 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec127dd-efc8-44c1-aebc-7d3826c4750b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:58.876506 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec127dd-efc8-44c1-aebc-7d3826c4750b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:58.876795 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec127dd-efc8-44c1-aebc-7d3826c4750b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:58.877076 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec127dd-efc8-44c1-aebc-7d3826c4750b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:58.887971 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec127dd-efc8-44c1-aebc-7d3826c4750b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:58.889493 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec127dd-efc8-44c1-aebc-7d3826c4750b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:58.908147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-0ec127dd-efc8-44c1-aebc-7d3826c4750b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:58.908147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:58.908147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:58.908147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:58.908147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:58.908147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:58.908147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:58.908147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:58.908147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:58.908147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:58.908147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:58.908147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:58.908147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:51:58.908147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:51:58.908147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:51:58.909148 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:51:58.909148 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:58.909148 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:58.909148 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:58.909148 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:58.909148 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:58.909148 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:58.909148 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:58.909148 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:58.909148 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:58.909148 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:58.909148 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:58.909148 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:58.909148 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:58.909148 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:58.909148 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:58.909148 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:58.910218 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:51:58.910218 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:51:58.910218 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:51:58.910218 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:51:58.914011 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 415/851] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:51:58.915463 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec127dd-efc8-44c1-aebc-7d3826c4750b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:58.956761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf5235-8976-47b0-96b2-358ab9568bcc None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:58.965223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:59.004930 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5915ad-538f-4505-81fa-5b6aff33188a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:59.006810 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5915ad-538f-4505-81fa-5b6aff33188a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:59.007633 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5915ad-538f-4505-81fa-5b6aff33188a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:59.008280 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5915ad-538f-4505-81fa-5b6aff33188a None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:59.009240 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5915ad-538f-4505-81fa-5b6aff33188a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:59.009798 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5915ad-538f-4505-81fa-5b6aff33188a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:59.011529 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5915ad-538f-4505-81fa-5b6aff33188a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:59.018308 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 437/852] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jul 21 15:51:58 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:51:59.032360 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec127dd-efc8-44c1-aebc-7d3826c4750b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:59.074432 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575cedf0-4cb3-45b4-a712-9d0c454db339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:59.075486 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575cedf0-4cb3-45b4-a712-9d0c454db339 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:59.075972 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575cedf0-4cb3-45b4-a712-9d0c454db339 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:59.076535 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575cedf0-4cb3-45b4-a712-9d0c454db339 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:59.077677 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575cedf0-4cb3-45b4-a712-9d0c454db339 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:59.078214 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575cedf0-4cb3-45b4-a712-9d0c454db339 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:59.079897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575cedf0-4cb3-45b4-a712-9d0c454db339 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:59.101844 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 416/853] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:51:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:59.111273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5915ad-538f-4505-81fa-5b6aff33188a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:59.116435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdbfc8-c989-4ab4-bd33-2ed3796f5097 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:59.116718 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdbfc8-c989-4ab4-bd33-2ed3796f5097 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:59.118174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdbfc8-c989-4ab4-bd33-2ed3796f5097 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:59.118756 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 417/854] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 15:51:59.125084 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006641bd-df11-43c7-aed8-d840d1ee6bb5 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:59.125372 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006641bd-df11-43c7-aed8-d840d1ee6bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:59.125687 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006641bd-df11-43c7-aed8-d840d1ee6bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:59.156564 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-006641bd-df11-43c7-aed8-d840d1ee6bb5 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:59.159820 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-006641bd-df11-43c7-aed8-d840d1ee6bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:59.170382 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:59.257621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 418/855] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 15:51:59.258776 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c444934-25d8-4314-8d4a-a637da0ab9f3 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:59.259147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c444934-25d8-4314-8d4a-a637da0ab9f3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:59.259447 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c444934-25d8-4314-8d4a-a637da0ab9f3 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:59.264696 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 419/856] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:51:59.267684 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257ea55-f4dc-4bb0-9e4e-ccfd5cd2be4f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:59.267974 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257ea55-f4dc-4bb0-9e4e-ccfd5cd2be4f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:59.268257 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257ea55-f4dc-4bb0-9e4e-ccfd5cd2be4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:59.276689 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5915ad-538f-4505-81fa-5b6aff33188a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:59.322666 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5a6a9-2e0d-4fe3-8bf4-30c75066c3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:59.323482 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5a6a9-2e0d-4fe3-8bf4-30c75066c3b2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:59.323763 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5a6a9-2e0d-4fe3-8bf4-30c75066c3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:59.324032 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5a6a9-2e0d-4fe3-8bf4-30c75066c3b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:59.332361 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5a6a9-2e0d-4fe3-8bf4-30c75066c3b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:59.333834 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5a6a9-2e0d-4fe3-8bf4-30c75066c3b2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:59.353229 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-abe5a6a9-2e0d-4fe3-8bf4-30c75066c3b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:59.353229 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:59.353229 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:59.353229 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:59.353229 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:59.353229 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:59.353229 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:59.353229 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:59.353229 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:59.353229 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:59.353229 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:59.353229 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:59.353229 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:51:59.353229 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:51:59.353229 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:51:59.354440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:51:59.354440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:59.354440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:59.354440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:59.354440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:59.354440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:59.354440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:59.354440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:59.354440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:59.354440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:59.354440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:59.354440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:59.354440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:59.354440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:59.354440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:59.354440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:59.354440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:59.355357 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:51:59.355357 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:51:59.355357 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:51:59.355357 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:51:59.355357 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:51:59.355357 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:51:59.355357 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:51:59.355357 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:59.358774 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 438/857] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:51:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:59.360328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5a6a9-2e0d-4fe3-8bf4-30c75066c3b2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:59.411849 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1257ea55-f4dc-4bb0-9e4e-ccfd5cd2be4f None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:51:59.413097 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1257ea55-f4dc-4bb0-9e4e-ccfd5cd2be4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:51:59.443176 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:59.491947 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:59.494201 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 420/858] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:51:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:51:59.498815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1257ea55-f4dc-4bb0-9e4e-ccfd5cd2be4f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:59.500637 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5a6a9-2e0d-4fe3-8bf4-30c75066c3b2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:59.541476 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae34d8f4-4e6b-400e-a539-f4611e4cc29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:59.542438 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34d8f4-4e6b-400e-a539-f4611e4cc29a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:59.542723 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34d8f4-4e6b-400e-a539-f4611e4cc29a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:59.543002 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34d8f4-4e6b-400e-a539-f4611e4cc29a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:59.555477 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae34d8f4-4e6b-400e-a539-f4611e4cc29a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:51:59.564610 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-ae34d8f4-4e6b-400e-a539-f4611e4cc29a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:59.564610 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:51:59.564610 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:51:59.564610 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:51:59.564610 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:51:59.564610 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:51:59.564610 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:51:59.564610 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:51:59.564610 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:51:59.564610 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:51:59.564610 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:51:59.564610 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:51:59.564610 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:51:59.564610 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:51:59.564610 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:51:59.564610 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:51:59.566604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:51:59.566604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:51:59.566604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:51:59.566604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:51:59.566604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:51:59.566604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:51:59.566604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:51:59.566604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:51:59.566604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:51:59.566604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:51:59.566604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:51:59.566604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:51:59.566604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:51:59.566604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:51:59.566604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:51:59.566604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:51:59.567495 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:51:59.567495 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:51:59.567495 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:51:59.567495 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:51:59.567495 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:51:59.567495 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:51:59.567495 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:51:59.567495 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:51:59.567495 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:51:59.567495 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:51:59.567495 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:51:59.567495 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:51:59.567495 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 439/859] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 15:51:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:51:59.567495 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae34d8f4-4e6b-400e-a539-f4611e4cc29a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:59.642664 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1257ea55-f4dc-4bb0-9e4e-ccfd5cd2be4f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:59.683693 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c737f-5847-4ee0-823d-3e97bdebde7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:59.684617 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c737f-5847-4ee0-823d-3e97bdebde7f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:59.685075 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c737f-5847-4ee0-823d-3e97bdebde7f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:59.685518 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c737f-5847-4ee0-823d-3e97bdebde7f None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:59.686539 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c737f-5847-4ee0-823d-3e97bdebde7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:59.687012 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c737f-5847-4ee0-823d-3e97bdebde7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:59.687654 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae34d8f4-4e6b-400e-a539-f4611e4cc29a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:59.688829 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c737f-5847-4ee0-823d-3e97bdebde7f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:59.701331 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 421/860] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:51:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:59.702745 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c737f-5847-4ee0-823d-3e97bdebde7f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:59.733694 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b2853b-43d7-45b9-b2d2-ab68965117dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:59.733935 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:59.734363 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b2853b-43d7-45b9-b2d2-ab68965117dc None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:59.735113 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b2853b-43d7-45b9-b2d2-ab68965117dc None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:59.735435 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b2853b-43d7-45b9-b2d2-ab68965117dc None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:59.736097 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b2853b-43d7-45b9-b2d2-ab68965117dc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:59.737523 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b2853b-43d7-45b9-b2d2-ab68965117dc None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:59.757303 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 440/861] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 21 15:51:59 2022] POST /identity/v3/services => generated 251 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:51:59.758166 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b2853b-43d7-45b9-b2d2-ab68965117dc None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:59.795601 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:51:59.863188 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c737f-5847-4ee0-823d-3e97bdebde7f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:59.908939 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b2853b-43d7-45b9-b2d2-ab68965117dc None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:51:59.919433 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ec74a-230f-4d29-91b3-8e0587b1d725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:59.921427 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ec74a-230f-4d29-91b3-8e0587b1d725 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:59.921909 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ec74a-230f-4d29-91b3-8e0587b1d725 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:59.922680 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ec74a-230f-4d29-91b3-8e0587b1d725 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:59.923931 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357ec74a-230f-4d29-91b3-8e0587b1d725 None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:59.924408 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357ec74a-230f-4d29-91b3-8e0587b1d725 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:59.927421 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357ec74a-230f-4d29-91b3-8e0587b1d725 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:59.940452 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 422/862] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 15:51:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:51:59.946367 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ec74a-230f-4d29-91b3-8e0587b1d725 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:51:59.953382 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72a373-0898-4a13-9e1e-368d7942d7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:51:59.954353 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72a373-0898-4a13-9e1e-368d7942d7b9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:51:59.954830 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72a373-0898-4a13-9e1e-368d7942d7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:51:59.955264 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72a373-0898-4a13-9e1e-368d7942d7b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:51:59.956298 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a72a373-0898-4a13-9e1e-368d7942d7b9 None admin] RBAC: Adding query filter params (role.id=475521b8a0174037b162f92022aea1a1, scope.project.id=bb1ac00308714c39a79aae09fbb73021, user.id=f5bfae2cb30843b1b1a0343efef38638) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:51:59.957722 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a72a373-0898-4a13-9e1e-368d7942d7b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:51:59.959618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a72a373-0898-4a13-9e1e-368d7942d7b9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:51:59.970883 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 441/863] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:51:59 2022] GET /identity/v3/role_assignments?user.id=f5bfae2cb30843b1b1a0343efef38638&scope.project.id=bb1ac00308714c39a79aae09fbb73021&role.id=475521b8a0174037b162f92022aea1a1 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:00.065132 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:00.082524 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ec74a-230f-4d29-91b3-8e0587b1d725 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:00.122008 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbebfe-e777-416f-b6e5-137b119e2971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:00.122913 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbebfe-e777-416f-b6e5-137b119e2971 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:00.123266 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbebfe-e777-416f-b6e5-137b119e2971 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:00.123649 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbebfe-e777-416f-b6e5-137b119e2971 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:00.135869 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfbebfe-e777-416f-b6e5-137b119e2971 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:00.137443 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfbebfe-e777-416f-b6e5-137b119e2971 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:00.157043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-2bfbebfe-e777-416f-b6e5-137b119e2971 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:52:00.157043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:00.157043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:00.157043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:00.157043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:00.157043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:00.157043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:00.157043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:00.157043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:00.157043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:00.157043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:00.157043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:00.157043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:52:00.157043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:52:00.157043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:52:00.158278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:52:00.158278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:00.158278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:00.158278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:00.158278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:00.158278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:00.158278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:00.158278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:00.158278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:00.158278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:00.158278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:00.158278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:00.158278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:00.158278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:00.158278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:00.158278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:00.158278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:00.159312 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:52:00.159312 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:52:00.159312 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:52:00.159312 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:00.160655 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 423/864] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:51:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:00.164967 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72a373-0898-4a13-9e1e-368d7942d7b9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:00.269991 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:00.292091 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72a373-0898-4a13-9e1e-368d7942d7b9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:00.342839 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a0872-fd36-4d49-a0db-e2bb6c7e6edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:00.343636 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a0872-fd36-4d49-a0db-e2bb6c7e6edf None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:00.343919 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a0872-fd36-4d49-a0db-e2bb6c7e6edf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:00.344236 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a0872-fd36-4d49-a0db-e2bb6c7e6edf None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:00.345119 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a0872-fd36-4d49-a0db-e2bb6c7e6edf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:00.345513 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a0872-fd36-4d49-a0db-e2bb6c7e6edf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:00.348159 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a0872-fd36-4d49-a0db-e2bb6c7e6edf None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:00.366128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 442/865] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:52:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:00.374808 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbebfe-e777-416f-b6e5-137b119e2971 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:00.481198 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbebfe-e777-416f-b6e5-137b119e2971 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:00.517738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c5d4a1-8427-40ce-a709-92cbac239f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:00.518558 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5d4a1-8427-40ce-a709-92cbac239f2c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:00.518944 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5d4a1-8427-40ce-a709-92cbac239f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:00.519229 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5d4a1-8427-40ce-a709-92cbac239f2c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:00.528049 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c5d4a1-8427-40ce-a709-92cbac239f2c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:00.529503 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c5d4a1-8427-40ce-a709-92cbac239f2c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:00.545246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-e8c5d4a1-8427-40ce-a709-92cbac239f2c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 15:52:00.545246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:00.545246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:00.545246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:00.545246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:00.545246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:00.545246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:00.545246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:00.545246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:00.545246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:00.545246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:00.545246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:00.545246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:52:00.545246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:52:00.545246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:52:00.545246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:52:00.548063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:00.548063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:00.548063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:52:00.548063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:52:00.548063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:52:00.548063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:52:00.548063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:00.548063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:00.548063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:00.548063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:00.548063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:00.548063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:00.548063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:00.548063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:00.548063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:00.548063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:00.548063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:00.548063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:00.552306 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:00.552306 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:00.552306 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:52:00.552306 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:52:00.552306 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:52:00.552306 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:52:00.552306 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:52:00.552306 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:52:00.552306 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 15:52:00.552306 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:00.552306 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:00.552306 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 424/866] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jul 21 15:52:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:00.558417 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a0872-fd36-4d49-a0db-e2bb6c7e6edf None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:00.671898 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a0872-fd36-4d49-a0db-e2bb6c7e6edf None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:00.711082 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8299496c-4ea5-41b7-a063-8bb90639be5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:00.711761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299496c-4ea5-41b7-a063-8bb90639be5f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:00.712046 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299496c-4ea5-41b7-a063-8bb90639be5f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:00.712328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299496c-4ea5-41b7-a063-8bb90639be5f None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:00.713186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8299496c-4ea5-41b7-a063-8bb90639be5f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:00.713502 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8299496c-4ea5-41b7-a063-8bb90639be5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:00.714989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8299496c-4ea5-41b7-a063-8bb90639be5f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:00.718123 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:00.744116 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 443/867] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Jul 21 15:52:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:00.750336 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c5d4a1-8427-40ce-a709-92cbac239f2c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:00.858860 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c5d4a1-8427-40ce-a709-92cbac239f2c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:00.895424 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec2516-8c17-407d-93eb-defef064a05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:00.896464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec2516-8c17-407d-93eb-defef064a05f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:00.896974 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec2516-8c17-407d-93eb-defef064a05f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:00.897447 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec2516-8c17-407d-93eb-defef064a05f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:00.966315 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:00.970788 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dec2516-8c17-407d-93eb-defef064a05f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:00.972688 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dec2516-8c17-407d-93eb-defef064a05f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:01.081774 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 425/868] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:52:00 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 15:52:01.086717 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8299496c-4ea5-41b7-a063-8bb90639be5f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:01.153895 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd53e7-2392-4019-ac43-79a9830247b4 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:01.154190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd53e7-2392-4019-ac43-79a9830247b4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:01.154490 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd53e7-2392-4019-ac43-79a9830247b4 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:01.156302 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 426/869] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 15:52:01.162549 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e4c6b-6f16-4719-911b-0d57560034bb None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:01.162980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e4c6b-6f16-4719-911b-0d57560034bb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:01.163349 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e4c6b-6f16-4719-911b-0d57560034bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:01.182762 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-cb8e4c6b-6f16-4719-911b-0d57560034bb None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:01.184281 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cb8e4c6b-6f16-4719-911b-0d57560034bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:01.205909 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8299496c-4ea5-41b7-a063-8bb90639be5f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:01.255223 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98bbb6a-9c53-4a0e-95d8-f2979b9cfe59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:01.256376 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98bbb6a-9c53-4a0e-95d8-f2979b9cfe59 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:01.256970 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98bbb6a-9c53-4a0e-95d8-f2979b9cfe59 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:01.257564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98bbb6a-9c53-4a0e-95d8-f2979b9cfe59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:01.281632 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 427/870] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:01 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:52:01.286099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98bbb6a-9c53-4a0e-95d8-f2979b9cfe59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:01.287578 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98bbb6a-9c53-4a0e-95d8-f2979b9cfe59 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:01.302682 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e4c6b-6f16-4719-911b-0d57560034bb None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:01.307334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d98bbb6a-9c53-4a0e-95d8-f2979b9cfe59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:52:01.307334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:01.307334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:01.307334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:01.307334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:01.307334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:01.307334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:01.307334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:01.307334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:01.307334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:01.307334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:01.307334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:01.307334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:52:01.307334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:52:01.307334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:52:01.308250 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:52:01.308250 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:01.308250 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:01.308250 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:01.308250 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:01.308250 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:01.308250 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:01.308250 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:01.308250 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:01.308250 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:01.308250 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:01.308250 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:01.308250 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:01.308250 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:01.308250 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:01.308250 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:01.308250 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:01.309205 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:52:01.309205 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:52:01.309205 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:52:01.309205 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:01.312572 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 444/871] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:52:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:01.319331 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98bbb6a-9c53-4a0e-95d8-f2979b9cfe59 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:01.340882 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:01.376847 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:01.449272 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98bbb6a-9c53-4a0e-95d8-f2979b9cfe59 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:01.463217 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e4c6b-6f16-4719-911b-0d57560034bb None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:01.490894 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796fa51d-d5d7-4ba7-8665-d4a4f058ff86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:01.491924 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796fa51d-d5d7-4ba7-8665-d4a4f058ff86 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:01.492486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796fa51d-d5d7-4ba7-8665-d4a4f058ff86 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:01.492975 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796fa51d-d5d7-4ba7-8665-d4a4f058ff86 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:01.494196 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796fa51d-d5d7-4ba7-8665-d4a4f058ff86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:01.494714 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796fa51d-d5d7-4ba7-8665-d4a4f058ff86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:01.496456 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796fa51d-d5d7-4ba7-8665-d4a4f058ff86 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:01.512419 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b28262-fa9a-4ce9-af80-6505cbe427b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:01.513090 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b28262-fa9a-4ce9-af80-6505cbe427b6 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:01.513370 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b28262-fa9a-4ce9-af80-6505cbe427b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:01.513671 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b28262-fa9a-4ce9-af80-6505cbe427b6 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:01.514454 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b28262-fa9a-4ce9-af80-6505cbe427b6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:01.519678 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b28262-fa9a-4ce9-af80-6505cbe427b6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:01.520510 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 445/872] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:01.523342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796fa51d-d5d7-4ba7-8665-d4a4f058ff86 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:01.537839 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 428/873] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 21 15:52:01 2022] POST /identity/v3/services => generated 269 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 15:52:01.659272 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796fa51d-d5d7-4ba7-8665-d4a4f058ff86 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:01.669527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:01.708212 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9f082-dd83-4b47-816a-717e679eaf6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:01.709837 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9f082-dd83-4b47-816a-717e679eaf6c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:01.710326 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9f082-dd83-4b47-816a-717e679eaf6c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:01.710801 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9f082-dd83-4b47-816a-717e679eaf6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:01.718995 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f9f082-dd83-4b47-816a-717e679eaf6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:01.720779 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f9f082-dd83-4b47-816a-717e679eaf6c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:01.736890 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-c1f9f082-dd83-4b47-816a-717e679eaf6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:52:01.736890 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:01.736890 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:01.736890 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:01.736890 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:01.736890 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:01.736890 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:01.736890 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:01.736890 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:01.736890 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:01.736890 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:01.736890 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:01.736890 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:52:01.736890 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:52:01.736890 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:52:01.738048 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:52:01.738048 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:01.738048 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:01.738048 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:01.738048 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:01.738048 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:01.738048 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:01.738048 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:01.738048 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:01.738048 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:01.738048 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:01.738048 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:01.738048 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:01.738048 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:01.738048 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:01.738048 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:01.738048 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:01.739186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:52:01.739186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:52:01.739186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:52:01.739186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:52:01.739186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:52:01.739186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:52:01.739186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:52:01.739186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:01.741224 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 446/874] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:52:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:01.748265 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b28262-fa9a-4ce9-af80-6505cbe427b6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:01.862308 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:01.876513 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b28262-fa9a-4ce9-af80-6505cbe427b6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:01.915602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad54c6-a1fe-4eb8-8253-260b99da1a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:01.916495 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad54c6-a1fe-4eb8-8253-260b99da1a8f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:01.916931 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad54c6-a1fe-4eb8-8253-260b99da1a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:01.917354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad54c6-a1fe-4eb8-8253-260b99da1a8f None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:01.918352 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ad54c6-a1fe-4eb8-8253-260b99da1a8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:01.918841 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ad54c6-a1fe-4eb8-8253-260b99da1a8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:01.920497 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ad54c6-a1fe-4eb8-8253-260b99da1a8f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:01.934240 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 429/875] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:52:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:01.940580 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9f082-dd83-4b47-816a-717e679eaf6c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:02.064965 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9f082-dd83-4b47-816a-717e679eaf6c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:02.106783 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb226b-b5fa-47d9-a40b-a0a9135c8631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:02.107894 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb226b-b5fa-47d9-a40b-a0a9135c8631 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:02.108689 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb226b-b5fa-47d9-a40b-a0a9135c8631 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:02.108975 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb226b-b5fa-47d9-a40b-a0a9135c8631 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:02.109872 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddb226b-b5fa-47d9-a40b-a0a9135c8631 None admin] RBAC: Adding query filter params (role.id=64d94d0d5a7640818c21aa5a622624ab, scope.project.id=bb1ac00308714c39a79aae09fbb73021, user.id=6edbad98ed0f4823b9955f9f0e270149) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:02.110242 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddb226b-b5fa-47d9-a40b-a0a9135c8631 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:02.111745 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddb226b-b5fa-47d9-a40b-a0a9135c8631 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:02.120122 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 447/876] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:52:01 2022] GET /identity/v3/role_assignments?user.id=6edbad98ed0f4823b9955f9f0e270149&scope.project.id=bb1ac00308714c39a79aae09fbb73021&role.id=64d94d0d5a7640818c21aa5a622624ab => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:02.312664 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2584c-18b9-4f1d-a5f9-23fc68b81dbf None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:02.313116 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2584c-18b9-4f1d-a5f9-23fc68b81dbf None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:02.313501 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2584c-18b9-4f1d-a5f9-23fc68b81dbf None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:02.315308 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 430/877] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:52:02.321681 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25873e0-5e4b-4684-a2c5-e9351cf1219c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:02.322213 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25873e0-5e4b-4684-a2c5-e9351cf1219c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:02.322794 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25873e0-5e4b-4684-a2c5-e9351cf1219c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:02.346962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d25873e0-5e4b-4684-a2c5-e9351cf1219c None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:02.348433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d25873e0-5e4b-4684-a2c5-e9351cf1219c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:02.372067 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:02.430032 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 448/878] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:02 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:52:02.433147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc71072-be26-4e48-be10-bc5cd297eed6 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:02.433535 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc71072-be26-4e48-be10-bc5cd297eed6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:02.433942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc71072-be26-4e48-be10-bc5cd297eed6 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:02.435642 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 431/879] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:52:02.441172 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637d0bf-4788-436d-93aa-8c2877544154 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:02.441475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637d0bf-4788-436d-93aa-8c2877544154 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:02.441833 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637d0bf-4788-436d-93aa-8c2877544154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:02.456371 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc71072-be26-4e48-be10-bc5cd297eed6 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:02.460722 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8637d0bf-4788-436d-93aa-8c2877544154 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:02.461954 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8637d0bf-4788-436d-93aa-8c2877544154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:02.552719 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 449/880] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:02 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:52:02.573801 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637d0bf-4788-436d-93aa-8c2877544154 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:02.603624 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc71072-be26-4e48-be10-bc5cd297eed6 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:02.655304 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba085f0-5a92-4b7c-8f94-f59072d9f8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:02.656157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba085f0-5a92-4b7c-8f94-f59072d9f8d8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:02.656470 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba085f0-5a92-4b7c-8f94-f59072d9f8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:02.656766 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba085f0-5a92-4b7c-8f94-f59072d9f8d8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:02.657435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba085f0-5a92-4b7c-8f94-f59072d9f8d8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:02.659180 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba085f0-5a92-4b7c-8f94-f59072d9f8d8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:02.674093 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-aba085f0-5a92-4b7c-8f94-f59072d9f8d8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 15:52:02.674093 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:02.674093 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 15:52:02.674093 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 15:52:02.674093 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 15:52:02.674093 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 15:52:02.674093 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 15:52:02.674093 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:52:02.674093 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 15:52:02.674093 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:02.674093 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 15:52:02.674093 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:02.674093 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:02.674093 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:02.674093 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:02.674093 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:02.674093 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:02.675204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:02.675204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:02.675204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:02.675204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:02.675204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:02.675204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:02.675204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 15:52:02.675204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 15:52:02.675204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 15:52:02.675204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 15:52:02.675204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:02.675204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:02.675204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:02.675204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:02.675204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:02.675204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:02.676097 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:02.676097 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:02.676097 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:02.676097 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:02.676097 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:02.676097 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:02.676097 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:02.676097 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:02.676097 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:02.676097 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:02.676097 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 15:52:02.676097 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:52:02.676097 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 15:52:02.676097 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:02.676097 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 432/881] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 15:52:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:52:02.681126 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:02.682806 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba085f0-5a92-4b7c-8f94-f59072d9f8d8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:02.699449 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637d0bf-4788-436d-93aa-8c2877544154 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:02.700185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:02.745211 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1174f162-f2fe-47c0-9c53-484ce6126569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:02.746124 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1174f162-f2fe-47c0-9c53-484ce6126569 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:02.746387 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1174f162-f2fe-47c0-9c53-484ce6126569 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:02.746616 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1174f162-f2fe-47c0-9c53-484ce6126569 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:02.747279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1174f162-f2fe-47c0-9c53-484ce6126569 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:02.750062 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1174f162-f2fe-47c0-9c53-484ce6126569 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:02.768004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-1174f162-f2fe-47c0-9c53-484ce6126569 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 15:52:02.768004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:02.768004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 15:52:02.768004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 15:52:02.768004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 15:52:02.768004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 15:52:02.768004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 15:52:02.768004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:52:02.768004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 15:52:02.768004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:02.768004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 15:52:02.768004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:02.768004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:02.768004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:02.768004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:02.768004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:02.768004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:02.769099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:02.769099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:02.769099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:02.769099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:02.769099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:02.769099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:02.769099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 15:52:02.769099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 15:52:02.769099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 15:52:02.769099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 15:52:02.769099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:02.769099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:02.769099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:02.769099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:02.769099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:02.769099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:02.770025 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:02.770025 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:02.770025 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:02.770025 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:02.770025 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:02.770025 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:02.770025 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:02.770025 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:02.770025 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:02.770025 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:02.770025 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 15:52:02.770025 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:52:02.770025 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 15:52:02.770025 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:02.773326 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 450/882] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jul 21 15:52:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:52:02.779041 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1174f162-f2fe-47c0-9c53-484ce6126569 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:02.826487 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba085f0-5a92-4b7c-8f94-f59072d9f8d8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:02.869705 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca898d11-4adf-4404-879b-b371157d0885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:02.870787 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca898d11-4adf-4404-879b-b371157d0885 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:02.871223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca898d11-4adf-4404-879b-b371157d0885 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:02.871758 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca898d11-4adf-4404-879b-b371157d0885 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:02.872621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca898d11-4adf-4404-879b-b371157d0885 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:02.873085 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca898d11-4adf-4404-879b-b371157d0885 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:02.875216 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca898d11-4adf-4404-879b-b371157d0885 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:02.881279 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 433/883] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 15:52:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:02.886838 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca898d11-4adf-4404-879b-b371157d0885 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:02.921149 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1174f162-f2fe-47c0-9c53-484ce6126569 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:02.960360 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da5802-953f-4bc1-bc66-7f7557707e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:02.961455 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da5802-953f-4bc1-bc66-7f7557707e96 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:02.962019 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da5802-953f-4bc1-bc66-7f7557707e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:02.962596 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da5802-953f-4bc1-bc66-7f7557707e96 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:02.963665 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62da5802-953f-4bc1-bc66-7f7557707e96 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:02.964181 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62da5802-953f-4bc1-bc66-7f7557707e96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:02.965843 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62da5802-953f-4bc1-bc66-7f7557707e96 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:02.977537 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:02.977892 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 451/884] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jul 21 15:52:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:52:02.979484 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da5802-953f-4bc1-bc66-7f7557707e96 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:02.999164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:03.006459 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca898d11-4adf-4404-879b-b371157d0885 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:03.049808 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821971fb-6329-45c5-b9ac-bee42d39da5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:03.050930 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821971fb-6329-45c5-b9ac-bee42d39da5e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:03.051322 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821971fb-6329-45c5-b9ac-bee42d39da5e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:03.051781 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821971fb-6329-45c5-b9ac-bee42d39da5e None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:03.053020 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821971fb-6329-45c5-b9ac-bee42d39da5e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:03.053511 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821971fb-6329-45c5-b9ac-bee42d39da5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:03.055397 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821971fb-6329-45c5-b9ac-bee42d39da5e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:03.066143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 434/885] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 15:52:02 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:52:03.067585 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821971fb-6329-45c5-b9ac-bee42d39da5e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:03.115628 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da5802-953f-4bc1-bc66-7f7557707e96 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:03.163580 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf9382-628b-47a8-b30d-36c6f0746a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:03.164265 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf9382-628b-47a8-b30d-36c6f0746a76 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:03.164555 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf9382-628b-47a8-b30d-36c6f0746a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:03.164835 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf9382-628b-47a8-b30d-36c6f0746a76 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:03.165562 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf9382-628b-47a8-b30d-36c6f0746a76 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:03.165902 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf9382-628b-47a8-b30d-36c6f0746a76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:03.167309 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf9382-628b-47a8-b30d-36c6f0746a76 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:03.181612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 452/886] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jul 21 15:52:02 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:03.219910 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821971fb-6329-45c5-b9ac-bee42d39da5e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:03.268002 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144b59ef-e1b8-4344-83e5-a8c65bd826bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:03.268739 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b59ef-e1b8-4344-83e5-a8c65bd826bf None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:03.269019 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b59ef-e1b8-4344-83e5-a8c65bd826bf None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:03.269292 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b59ef-e1b8-4344-83e5-a8c65bd826bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:03.270202 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144b59ef-e1b8-4344-83e5-a8c65bd826bf None admin] RBAC: Adding query filter params (interface=public, service_id=583b7dea904a4f3583873cb82014a9af, region_id=RegionOne) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:03.270517 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144b59ef-e1b8-4344-83e5-a8c65bd826bf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:03.271942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144b59ef-e1b8-4344-83e5-a8c65bd826bf None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:03.278727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 435/887] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Jul 21 15:52:03 2022] GET /identity/v3/endpoints?service_id=583b7dea904a4f3583873cb82014a9af&interface=public®ion_id=RegionOne => generated 194 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:03.284565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf9382-628b-47a8-b30d-36c6f0746a76 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:03.390867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:03.422246 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf9382-628b-47a8-b30d-36c6f0746a76 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:03.459816 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46795493-26c1-4cfb-b914-eac72e73e2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:03.460483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46795493-26c1-4cfb-b914-eac72e73e2fd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:03.460762 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46795493-26c1-4cfb-b914-eac72e73e2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:03.461037 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46795493-26c1-4cfb-b914-eac72e73e2fd None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:03.461748 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46795493-26c1-4cfb-b914-eac72e73e2fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:03.463295 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46795493-26c1-4cfb-b914-eac72e73e2fd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:03.470585 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 453/888] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jul 21 15:52:03 2022] GET /identity/v3/services => generated 1268 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 15:52:03.810217 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5c859-07c3-4f3d-8979-ad4b40331a14 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:03.811503 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5c859-07c3-4f3d-8979-ad4b40331a14 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:03.811503 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5c859-07c3-4f3d-8979-ad4b40331a14 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:03.813126 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 436/889] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 15:52:03.818862 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b49fc-f4bc-4866-93d5-5aec39b67943 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:03.819557 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b49fc-f4bc-4866-93d5-5aec39b67943 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:03.819557 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b49fc-f4bc-4866-93d5-5aec39b67943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:03.837351 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7e6b49fc-f4bc-4866-93d5-5aec39b67943 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:03.838613 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7e6b49fc-f4bc-4866-93d5-5aec39b67943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:03.938921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 454/890] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:03 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:52:03.965370 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5c859-07c3-4f3d-8979-ad4b40331a14 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:03.972388 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:04.102335 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5c859-07c3-4f3d-8979-ad4b40331a14 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:04.142432 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb26df-b42e-4d15-8180-082372c17911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:04.143609 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb26df-b42e-4d15-8180-082372c17911 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:04.144092 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb26df-b42e-4d15-8180-082372c17911 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:04.144528 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb26df-b42e-4d15-8180-082372c17911 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:04.145344 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bb26df-b42e-4d15-8180-082372c17911 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:04.146935 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bb26df-b42e-4d15-8180-082372c17911 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:04.160726 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-15bb26df-b42e-4d15-8180-082372c17911 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 15:52:04.160726 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:04.160726 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 15:52:04.160726 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 15:52:04.160726 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 15:52:04.160726 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 15:52:04.160726 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 15:52:04.160726 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:52:04.160726 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 15:52:04.160726 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:04.160726 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 15:52:04.160726 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:04.160726 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:04.160726 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:04.160726 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:04.160726 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:04.161989 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:04.161989 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:04.161989 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:04.161989 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:04.161989 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:04.161989 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:04.161989 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:04.161989 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 15:52:04.161989 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 15:52:04.161989 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 15:52:04.161989 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 15:52:04.161989 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:04.161989 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:04.161989 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:04.161989 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:04.161989 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:04.162924 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:04.162924 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:04.162924 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:04.162924 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:04.162924 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:04.162924 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:04.162924 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:04.162924 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:04.162924 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:04.162924 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:04.162924 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:04.162924 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 15:52:04.162924 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:52:04.162924 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 15:52:04.162924 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:04.165482 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 437/891] 10.222.0.52 () {56 vars in 1132 bytes} [Thu Jul 21 15:52:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:52:04.175040 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b49fc-f4bc-4866-93d5-5aec39b67943 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:04.257950 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:04.301702 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b49fc-f4bc-4866-93d5-5aec39b67943 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:04.346187 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8166bb9-9271-47fb-aec0-a155fe8e2885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:04.347424 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8166bb9-9271-47fb-aec0-a155fe8e2885 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:04.347717 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8166bb9-9271-47fb-aec0-a155fe8e2885 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:04.347995 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8166bb9-9271-47fb-aec0-a155fe8e2885 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:04.348729 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8166bb9-9271-47fb-aec0-a155fe8e2885 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:04.349039 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8166bb9-9271-47fb-aec0-a155fe8e2885 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:04.350966 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8166bb9-9271-47fb-aec0-a155fe8e2885 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:04.357037 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 455/892] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jul 21 15:52:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:04.365841 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb26df-b42e-4d15-8180-082372c17911 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:04.374473 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc147d01-d399-461b-98ee-20519372fd8b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:04.374757 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc147d01-d399-461b-98ee-20519372fd8b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:04.375029 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc147d01-d399-461b-98ee-20519372fd8b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:04.376701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 456/893] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 15:52:04.382300 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd7322-8a28-4d5c-8c19-6e7e9370930a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:04.382634 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd7322-8a28-4d5c-8c19-6e7e9370930a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:04.382971 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd7322-8a28-4d5c-8c19-6e7e9370930a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:04.403642 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b6fd7322-8a28-4d5c-8c19-6e7e9370930a None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:04.406099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b6fd7322-8a28-4d5c-8c19-6e7e9370930a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:04.456044 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:04.491058 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 457/894] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 15:52:04.505980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb26df-b42e-4d15-8180-082372c17911 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:04.515243 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd7322-8a28-4d5c-8c19-6e7e9370930a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:04.547968 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15012a1c-cffe-460d-8a84-4a43f57b0aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:04.548973 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15012a1c-cffe-460d-8a84-4a43f57b0aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:04.549391 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15012a1c-cffe-460d-8a84-4a43f57b0aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:04.549895 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15012a1c-cffe-460d-8a84-4a43f57b0aa5 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:04.550661 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15012a1c-cffe-460d-8a84-4a43f57b0aa5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:04.550976 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15012a1c-cffe-460d-8a84-4a43f57b0aa5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:04.552417 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15012a1c-cffe-460d-8a84-4a43f57b0aa5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:04.561986 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 438/895] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jul 21 15:52:04 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:04.567765 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15012a1c-cffe-460d-8a84-4a43f57b0aa5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:04.606032 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:04.655030 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd7322-8a28-4d5c-8c19-6e7e9370930a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:04.694381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06fa35-1962-4005-b7df-3a25ce04eb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:04.695272 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06fa35-1962-4005-b7df-3a25ce04eb3f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:04.695607 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06fa35-1962-4005-b7df-3a25ce04eb3f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:04.695905 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06fa35-1962-4005-b7df-3a25ce04eb3f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:04.707939 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06fa35-1962-4005-b7df-3a25ce04eb3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:52:04.709440 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15012a1c-cffe-460d-8a84-4a43f57b0aa5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:04.716389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-9b06fa35-1962-4005-b7df-3a25ce04eb3f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 15:52:04.716389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:04.716389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:04.716389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:04.716389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:04.716389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:04.716389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:04.716389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:04.716389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:04.716389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:04.716389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:04.716389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:04.716389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:52:04.716389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:52:04.716389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:52:04.716389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:52:04.717795 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:52:04.717795 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:52:04.717795 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:52:04.717795 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:52:04.717795 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:04.717795 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:04.717795 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:04.717795 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:04.717795 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:04.717795 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:04.717795 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:04.717795 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:04.717795 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:04.717795 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:04.717795 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:04.717795 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:04.718824 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:04.718824 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:04.718824 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:04.718824 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:04.718824 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:52:04.718824 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:52:04.718824 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:52:04.718824 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:52:04.718824 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:52:04.718824 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:52:04.718824 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 15:52:04.718824 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:04.723019 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 458/896] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Jul 21 15:52:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:04.723920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06fa35-1962-4005-b7df-3a25ce04eb3f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:04.752239 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:04.759278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671c3fe-f8aa-400d-83a5-e94110cfc1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:04.760907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671c3fe-f8aa-400d-83a5-e94110cfc1e5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:04.761209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671c3fe-f8aa-400d-83a5-e94110cfc1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:04.761506 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671c3fe-f8aa-400d-83a5-e94110cfc1e5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:04.762481 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e671c3fe-f8aa-400d-83a5-e94110cfc1e5 None admin] RBAC: Adding query filter params (interface=public, service_id=7ecf8ad606aa4d30b3b011c90a4c9687, region_id=RegionOne) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:04.762921 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e671c3fe-f8aa-400d-83a5-e94110cfc1e5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:04.766161 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e671c3fe-f8aa-400d-83a5-e94110cfc1e5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:04.777754 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 439/897] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Jul 21 15:52:04 2022] GET /identity/v3/endpoints?service_id=7ecf8ad606aa4d30b3b011c90a4c9687&interface=public®ion_id=RegionOne => generated 194 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:04.779846 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671c3fe-f8aa-400d-83a5-e94110cfc1e5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:04.858949 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06fa35-1962-4005-b7df-3a25ce04eb3f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:04.888148 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:04.898796 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd4109c-02f1-4be7-9789-2409ad7303fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:04.899566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4109c-02f1-4be7-9789-2409ad7303fe None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:04.899855 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4109c-02f1-4be7-9789-2409ad7303fe None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:04.900132 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4109c-02f1-4be7-9789-2409ad7303fe None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:04.900896 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd4109c-02f1-4be7-9789-2409ad7303fe None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:04.901204 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd4109c-02f1-4be7-9789-2409ad7303fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:04.902630 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd4109c-02f1-4be7-9789-2409ad7303fe None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:04.916967 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 459/898] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Jul 21 15:52:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:04.917934 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd4109c-02f1-4be7-9789-2409ad7303fe None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:04.919079 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671c3fe-f8aa-400d-83a5-e94110cfc1e5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:04.963764 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a404f76-dfff-409b-b646-2fd1215703c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:04.964439 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a404f76-dfff-409b-b646-2fd1215703c7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:04.964725 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a404f76-dfff-409b-b646-2fd1215703c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:04.965005 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a404f76-dfff-409b-b646-2fd1215703c7 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:04.965735 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a404f76-dfff-409b-b646-2fd1215703c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:04.967153 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a404f76-dfff-409b-b646-2fd1215703c7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:04.981606 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 440/899] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jul 21 15:52:04 2022] GET /identity/v3/services => generated 1268 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 15:52:05.063061 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd4109c-02f1-4be7-9789-2409ad7303fe None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:05.103466 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529529c-57c5-4993-a2e3-28efbc40f3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:05.104285 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529529c-57c5-4993-a2e3-28efbc40f3fe None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:05.104567 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529529c-57c5-4993-a2e3-28efbc40f3fe None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:05.104844 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529529c-57c5-4993-a2e3-28efbc40f3fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:05.117205 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f529529c-57c5-4993-a2e3-28efbc40f3fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:05.118734 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f529529c-57c5-4993-a2e3-28efbc40f3fe None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:05.141947 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-f529529c-57c5-4993-a2e3-28efbc40f3fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:52:05.141947 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:05.141947 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:05.141947 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:05.141947 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:05.141947 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:05.141947 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:05.141947 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:05.141947 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:05.141947 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:05.141947 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:05.141947 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:05.141947 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:52:05.141947 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:52:05.141947 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:52:05.143032 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:52:05.143032 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:05.143032 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:05.143032 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:05.143032 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:05.143032 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:05.143032 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:05.143032 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:05.143032 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:05.143032 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:05.143032 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:05.143032 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:05.143032 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:05.143032 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:05.143032 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:05.143032 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:05.143032 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:05.143961 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:52:05.143961 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:52:05.143961 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:52:05.143961 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:05.146520 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 460/900] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:52:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:05.152099 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a404f76-dfff-409b-b646-2fd1215703c7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:05.234063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:05.281496 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a404f76-dfff-409b-b646-2fd1215703c7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:05.317871 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff10ce3f-50a9-48c1-b806-48e9ff167f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:05.318593 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10ce3f-50a9-48c1-b806-48e9ff167f40 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:05.318890 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10ce3f-50a9-48c1-b806-48e9ff167f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:05.319180 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10ce3f-50a9-48c1-b806-48e9ff167f40 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:05.319975 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff10ce3f-50a9-48c1-b806-48e9ff167f40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:05.320290 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff10ce3f-50a9-48c1-b806-48e9ff167f40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:05.321698 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff10ce3f-50a9-48c1-b806-48e9ff167f40 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:05.335077 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 441/901] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:52:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:05.341152 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529529c-57c5-4993-a2e3-28efbc40f3fe None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:05.396376 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:05.468015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529529c-57c5-4993-a2e3-28efbc40f3fe None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:05.505358 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77becd74-c340-474a-a276-866cccc227c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:05.506139 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77becd74-c340-474a-a276-866cccc227c8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:05.506427 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77becd74-c340-474a-a276-866cccc227c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:05.506707 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77becd74-c340-474a-a276-866cccc227c8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:05.516102 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77becd74-c340-474a-a276-866cccc227c8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:05.518679 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77becd74-c340-474a-a276-866cccc227c8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:05.518925 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c911d2-b716-4acf-9c27-9ead217595b4 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:05.518925 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c911d2-b716-4acf-9c27-9ead217595b4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:05.519334 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c911d2-b716-4acf-9c27-9ead217595b4 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:05.520970 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 442/902] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 15:52:05.526918 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430055d-466a-4f15-b28d-e8f23d42f1a8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:05.527550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430055d-466a-4f15-b28d-e8f23d42f1a8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:05.528034 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430055d-466a-4f15-b28d-e8f23d42f1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:05.543149 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-77becd74-c340-474a-a276-866cccc227c8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 15:52:05.543149 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:05.543149 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:05.543149 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:05.543149 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:05.543149 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:05.543149 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:05.543149 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:05.543149 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:05.543149 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:05.543149 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:05.543149 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:05.543149 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:52:05.543149 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:52:05.543149 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:52:05.543149 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:52:05.544215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:05.544215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:05.544215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:52:05.544215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:52:05.544215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:52:05.544215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:52:05.544215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:05.544215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:05.544215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:05.544215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:05.544215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:05.544215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:05.544215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:05.544215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:05.544215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:05.544215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:05.544215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:05.544215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:05.545271 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:05.545271 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:05.545271 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:52:05.545271 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:52:05.545271 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:52:05.545271 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:52:05.545271 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:52:05.545271 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:52:05.545271 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 15:52:05.545271 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:05.545271 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 461/903] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jul 21 15:52:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:05.550283 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77becd74-c340-474a-a276-866cccc227c8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:05.550561 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7430055d-466a-4f15-b28d-e8f23d42f1a8 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:05.552010 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7430055d-466a-4f15-b28d-e8f23d42f1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:05.641597 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 443/904] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:05 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 15:52:05.658469 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7430055d-466a-4f15-b28d-e8f23d42f1a8 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:05.686636 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:05.691614 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77becd74-c340-474a-a276-866cccc227c8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:05.720838 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:05.743276 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943bfa07-e213-458c-9b56-d00377756212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:05.744525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943bfa07-e213-458c-9b56-d00377756212 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:05.745206 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943bfa07-e213-458c-9b56-d00377756212 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:05.745753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943bfa07-e213-458c-9b56-d00377756212 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:05.747715 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943bfa07-e213-458c-9b56-d00377756212 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:05.748258 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943bfa07-e213-458c-9b56-d00377756212 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:05.749961 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943bfa07-e213-458c-9b56-d00377756212 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:05.778391 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 462/905] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Jul 21 15:52:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:05.781984 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943bfa07-e213-458c-9b56-d00377756212 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:05.794652 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7430055d-466a-4f15-b28d-e8f23d42f1a8 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:05.843993 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b49f8d-c31a-4b41-9bcf-1d9a674cca12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:05.845374 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b49f8d-c31a-4b41-9bcf-1d9a674cca12 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:05.846088 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b49f8d-c31a-4b41-9bcf-1d9a674cca12 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:05.846610 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b49f8d-c31a-4b41-9bcf-1d9a674cca12 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:05.848199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b49f8d-c31a-4b41-9bcf-1d9a674cca12 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:05.850231 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b49f8d-c31a-4b41-9bcf-1d9a674cca12 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:05.870315 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 444/906] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 21 15:52:05 2022] POST /identity/v3/services => generated 241 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 15:52:05.871964 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6852aae-72b8-4bac-b334-d221ea1fd7e5 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:05.872250 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6852aae-72b8-4bac-b334-d221ea1fd7e5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:05.872524 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6852aae-72b8-4bac-b334-d221ea1fd7e5 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:05.875716 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 445/907] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:52:05.892305 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f15af-2ed2-4c62-9750-d7077ecc392f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:05.892305 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f15af-2ed2-4c62-9750-d7077ecc392f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:05.892305 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f15af-2ed2-4c62-9750-d7077ecc392f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:05.919382 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ee6f15af-2ed2-4c62-9750-d7077ecc392f None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:05.920730 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ee6f15af-2ed2-4c62-9750-d7077ecc392f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:05.937321 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943bfa07-e213-458c-9b56-d00377756212 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:06.008245 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9f0c8-e3fe-454a-86b8-6e17372f68b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:06.009955 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9f0c8-e3fe-454a-86b8-6e17372f68b6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:06.010438 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9f0c8-e3fe-454a-86b8-6e17372f68b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:06.010938 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9f0c8-e3fe-454a-86b8-6e17372f68b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:06.025609 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 446/908] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 15:52:06.048425 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6f15af-2ed2-4c62-9750-d7077ecc392f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:06.055958 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:06.106172 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:06.116000 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9f0c8-e3fe-454a-86b8-6e17372f68b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:06.117676 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9f0c8-e3fe-454a-86b8-6e17372f68b6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:06.197593 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6f15af-2ed2-4c62-9750-d7077ecc392f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:06.246732 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f42d2-a1d4-491f-b345-43f9f4586e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:06.247563 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f42d2-a1d4-491f-b345-43f9f4586e96 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:06.247846 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f42d2-a1d4-491f-b345-43f9f4586e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:06.248130 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f42d2-a1d4-491f-b345-43f9f4586e96 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:06.249020 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6f42d2-a1d4-491f-b345-43f9f4586e96 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:06.252322 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6f42d2-a1d4-491f-b345-43f9f4586e96 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:06.271091 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-5f6f42d2-a1d4-491f-b345-43f9f4586e96 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 15:52:06.271091 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:06.271091 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 15:52:06.271091 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 15:52:06.271091 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 15:52:06.271091 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 15:52:06.271091 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 15:52:06.271091 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:52:06.271091 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 15:52:06.271091 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:06.271091 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 15:52:06.271091 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:06.271091 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:06.271091 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:06.271091 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:06.271091 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:06.271091 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:06.272177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:06.272177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:06.272177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:06.272177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:06.272177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:06.272177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:06.272177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 15:52:06.272177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 15:52:06.272177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 15:52:06.272177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 15:52:06.272177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:06.272177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:06.272177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:06.272177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:06.272177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:06.272177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:06.273047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:06.273047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:06.273047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:06.273047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:06.273047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:06.273047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:06.273047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:06.273047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:06.273047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:06.273047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:06.273047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 15:52:06.273047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:52:06.273047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 15:52:06.273047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:06.277756 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 447/909] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 15:52:06 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:52:06.279283 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f42d2-a1d4-491f-b345-43f9f4586e96 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:06.286691 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 463/910] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:52:05 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 15:52:06.291914 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9f0c8-e3fe-454a-86b8-6e17372f68b6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:06.367094 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:06.418053 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f42d2-a1d4-491f-b345-43f9f4586e96 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:06.420872 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9f0c8-e3fe-454a-86b8-6e17372f68b6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:06.422090 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:06.463802 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a2e5a2-ff52-4686-b2f6-2e5e3981684a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:06.464620 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2e5a2-ff52-4686-b2f6-2e5e3981684a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:06.464898 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2e5a2-ff52-4686-b2f6-2e5e3981684a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:06.465195 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2e5a2-ff52-4686-b2f6-2e5e3981684a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:06.471083 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0895f19f-a8b8-4c03-bfdf-5f97bf00a76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:06.472865 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895f19f-a8b8-4c03-bfdf-5f97bf00a76b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:06.473697 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895f19f-a8b8-4c03-bfdf-5f97bf00a76b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:06.474472 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895f19f-a8b8-4c03-bfdf-5f97bf00a76b None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:06.476009 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0895f19f-a8b8-4c03-bfdf-5f97bf00a76b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:06.476817 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0895f19f-a8b8-4c03-bfdf-5f97bf00a76b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:06.477091 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a2e5a2-ff52-4686-b2f6-2e5e3981684a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:06.478612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a2e5a2-ff52-4686-b2f6-2e5e3981684a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:06.480428 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0895f19f-a8b8-4c03-bfdf-5f97bf00a76b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:06.486898 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 448/911] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 15:52:06 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:06.493012 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0895f19f-a8b8-4c03-bfdf-5f97bf00a76b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:06.499176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-65a2e5a2-ff52-4686-b2f6-2e5e3981684a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:52:06.499176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:06.499176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:06.499176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:06.499176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:06.499176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:06.499176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:06.499176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:06.499176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:06.499176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:06.499176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:06.499176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:06.499176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:52:06.499176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:52:06.499176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:52:06.500136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:52:06.500136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:06.500136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:06.500136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:06.500136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:06.500136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:06.500136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:06.500136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:06.500136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:06.500136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:06.500136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:06.500136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:06.500136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:06.500136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:06.500136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:06.500136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:06.500136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:06.501317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:52:06.501317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:52:06.501317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:52:06.501317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:06.501317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 464/912] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:52:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:52:06.506231 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a2e5a2-ff52-4686-b2f6-2e5e3981684a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:06.606377 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0895f19f-a8b8-4c03-bfdf-5f97bf00a76b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:06.629614 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a2e5a2-ff52-4686-b2f6-2e5e3981684a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:06.644140 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08e462-f15b-4601-8f56-dd3777dc3bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:06.645103 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08e462-f15b-4601-8f56-dd3777dc3bdb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:06.645559 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08e462-f15b-4601-8f56-dd3777dc3bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:06.646020 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08e462-f15b-4601-8f56-dd3777dc3bdb None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:06.646932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08e462-f15b-4601-8f56-dd3777dc3bdb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:06.647427 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08e462-f15b-4601-8f56-dd3777dc3bdb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:06.649050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08e462-f15b-4601-8f56-dd3777dc3bdb None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:06.660387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 449/913] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 15:52:06 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:06.661838 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:06.662968 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08e462-f15b-4601-8f56-dd3777dc3bdb None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:06.680911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec60b71-be03-43e8-9b60-c29b793924d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:06.681666 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec60b71-be03-43e8-9b60-c29b793924d9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:06.682038 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec60b71-be03-43e8-9b60-c29b793924d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:06.682380 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec60b71-be03-43e8-9b60-c29b793924d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:06.683137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec60b71-be03-43e8-9b60-c29b793924d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:06.683505 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec60b71-be03-43e8-9b60-c29b793924d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:06.684927 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec60b71-be03-43e8-9b60-c29b793924d9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:06.699085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 465/914] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:52:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:06.708660 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec60b71-be03-43e8-9b60-c29b793924d9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:06.724062 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:06.788817 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08e462-f15b-4601-8f56-dd3777dc3bdb None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:06.826169 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f650c9-13a4-444f-afa3-210ed2041585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:06.827147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f650c9-13a4-444f-afa3-210ed2041585 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:06.827610 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f650c9-13a4-444f-afa3-210ed2041585 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:06.828035 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f650c9-13a4-444f-afa3-210ed2041585 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:06.829007 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f650c9-13a4-444f-afa3-210ed2041585 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:06.830783 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f650c9-13a4-444f-afa3-210ed2041585 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:06.830984 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec60b71-be03-43e8-9b60-c29b793924d9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:06.858831 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 450/915] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jul 21 15:52:06 2022] POST /identity/v3/endpoints => generated 346 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 15:52:06.892086 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d576b4-960d-4ebe-94dd-e45c8312bbe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:06.893127 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d576b4-960d-4ebe-94dd-e45c8312bbe6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:06.893516 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d576b4-960d-4ebe-94dd-e45c8312bbe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:06.893929 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d576b4-960d-4ebe-94dd-e45c8312bbe6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:06.903175 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d576b4-960d-4ebe-94dd-e45c8312bbe6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:06.905000 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d576b4-960d-4ebe-94dd-e45c8312bbe6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:06.922632 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-38d576b4-960d-4ebe-94dd-e45c8312bbe6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:52:06.922632 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:06.922632 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:06.922632 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:06.922632 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:06.922632 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:06.922632 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:06.922632 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:06.922632 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:06.922632 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:06.922632 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:06.922632 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:06.922632 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:52:06.922632 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:52:06.922632 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:52:06.923829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:52:06.923829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:06.923829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:06.923829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:06.923829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:06.923829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:06.923829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:06.923829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:06.923829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:06.923829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:06.923829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:06.923829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:06.923829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:06.923829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:06.923829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:06.923829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:06.923829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:06.924916 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:52:06.924916 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:52:06.924916 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:52:06.924916 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:52:06.924916 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:52:06.924916 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:52:06.924916 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:52:06.924916 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:06.924916 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 466/916] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:52:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:52:06.932604 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f650c9-13a4-444f-afa3-210ed2041585 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:07.040736 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:07.057221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f650c9-13a4-444f-afa3-210ed2041585 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:07.102374 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a3dc6-1b30-43e1-b829-1f07815791c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:07.103035 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a3dc6-1b30-43e1-b829-1f07815791c5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:07.103318 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a3dc6-1b30-43e1-b829-1f07815791c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:07.103592 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a3dc6-1b30-43e1-b829-1f07815791c5 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:07.104399 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a3dc6-1b30-43e1-b829-1f07815791c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:07.104711 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a3dc6-1b30-43e1-b829-1f07815791c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:07.106190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a3dc6-1b30-43e1-b829-1f07815791c5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:07.121838 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 451/917] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:52:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:07.128479 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d576b4-960d-4ebe-94dd-e45c8312bbe6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:07.159829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:07.265181 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d576b4-960d-4ebe-94dd-e45c8312bbe6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:07.268717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03926785-e80e-41fb-be26-1083afef4bab None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:07.269009 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03926785-e80e-41fb-be26-1083afef4bab None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:07.269289 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03926785-e80e-41fb-be26-1083afef4bab None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:07.270985 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 452/918] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:52:07.276811 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640bac2f-7365-41c9-892a-ae7b2c8b663f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:07.277094 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640bac2f-7365-41c9-892a-ae7b2c8b663f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:07.277373 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640bac2f-7365-41c9-892a-ae7b2c8b663f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:07.297920 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-640bac2f-7365-41c9-892a-ae7b2c8b663f None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:07.299141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-640bac2f-7365-41c9-892a-ae7b2c8b663f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:07.311819 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea38585-c519-4431-a7b6-259df3c2e910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:07.313013 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea38585-c519-4431-a7b6-259df3c2e910 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:07.313454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea38585-c519-4431-a7b6-259df3c2e910 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:07.313909 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea38585-c519-4431-a7b6-259df3c2e910 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:07.315657 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea38585-c519-4431-a7b6-259df3c2e910 None admin] RBAC: Adding query filter params (role.id=abd0095f0bf64a0c8e694dbf50d6fd9a, scope.project.id=bb1ac00308714c39a79aae09fbb73021, user.id=6edbad98ed0f4823b9955f9f0e270149) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:07.316280 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea38585-c519-4431-a7b6-259df3c2e910 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:07.318311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea38585-c519-4431-a7b6-259df3c2e910 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:07.327194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 467/919] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:52:07 2022] GET /identity/v3/role_assignments?user.id=6edbad98ed0f4823b9955f9f0e270149&scope.project.id=bb1ac00308714c39a79aae09fbb73021&role.id=abd0095f0bf64a0c8e694dbf50d6fd9a => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:07.385431 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 453/920] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 15:52:07.409036 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea38585-c519-4431-a7b6-259df3c2e910 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:07.514962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:07.532798 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea38585-c519-4431-a7b6-259df3c2e910 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:07.574699 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f29bfc-78ab-42b5-9fcd-364b94040dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:07.576364 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f29bfc-78ab-42b5-9fcd-364b94040dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:07.577089 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f29bfc-78ab-42b5-9fcd-364b94040dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:07.577806 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f29bfc-78ab-42b5-9fcd-364b94040dd1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:07.578893 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f29bfc-78ab-42b5-9fcd-364b94040dd1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:07.581634 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f29bfc-78ab-42b5-9fcd-364b94040dd1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:07.603930 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-58f29bfc-78ab-42b5-9fcd-364b94040dd1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 15:52:07.603930 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:07.603930 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 15:52:07.603930 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 15:52:07.603930 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 15:52:07.603930 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 15:52:07.603930 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 15:52:07.603930 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:52:07.603930 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 15:52:07.603930 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:07.603930 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 15:52:07.603930 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:07.603930 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:07.603930 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:07.603930 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:07.603930 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:07.605158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:07.605158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:07.605158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:07.605158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:07.605158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:07.605158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:07.605158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:07.605158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 15:52:07.605158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 15:52:07.605158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 15:52:07.605158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 15:52:07.605158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:07.605158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:07.605158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:07.605158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:07.605158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:07.606340 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:07.606340 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:07.606340 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:07.606340 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:07.606340 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:07.606340 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:07.606340 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:07.606340 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:07.606340 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:07.606340 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:07.606340 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:07.606340 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 15:52:07.606340 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:52:07.606340 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 15:52:07.606340 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:07.609520 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 468/921] 10.222.0.52 () {56 vars in 1132 bytes} [Thu Jul 21 15:52:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:07.618739 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640bac2f-7365-41c9-892a-ae7b2c8b663f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:07.745726 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:07.778721 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640bac2f-7365-41c9-892a-ae7b2c8b663f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:07.822089 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a833d-e82c-4cdc-902d-4f7783cdc383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:07.822767 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a833d-e82c-4cdc-902d-4f7783cdc383 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:07.823051 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a833d-e82c-4cdc-902d-4f7783cdc383 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:07.823329 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a833d-e82c-4cdc-902d-4f7783cdc383 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:07.824049 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a833d-e82c-4cdc-902d-4f7783cdc383 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:07.824359 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a833d-e82c-4cdc-902d-4f7783cdc383 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:07.825773 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a833d-e82c-4cdc-902d-4f7783cdc383 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:07.831646 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 454/922] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jul 21 15:52:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:07.837254 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f29bfc-78ab-42b5-9fcd-364b94040dd1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:07.964151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f29bfc-78ab-42b5-9fcd-364b94040dd1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:08.001965 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c70dc2-b1cf-47c7-80cc-2f9522556a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:08.003022 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c70dc2-b1cf-47c7-80cc-2f9522556a2c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:08.003575 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c70dc2-b1cf-47c7-80cc-2f9522556a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:08.004147 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c70dc2-b1cf-47c7-80cc-2f9522556a2c None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:08.005290 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c70dc2-b1cf-47c7-80cc-2f9522556a2c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:08.007169 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c70dc2-b1cf-47c7-80cc-2f9522556a2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:08.008838 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c70dc2-b1cf-47c7-80cc-2f9522556a2c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:08.019739 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 469/923] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jul 21 15:52:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:08.026115 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a833d-e82c-4cdc-902d-4f7783cdc383 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:08.160615 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a833d-e82c-4cdc-902d-4f7783cdc383 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:08.198058 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da9e10-22da-4894-9292-da3d12194a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:08.198759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da9e10-22da-4894-9292-da3d12194a0b None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:08.199046 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da9e10-22da-4894-9292-da3d12194a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:08.199327 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da9e10-22da-4894-9292-da3d12194a0b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:08.200127 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da9e10-22da-4894-9292-da3d12194a0b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:08.202181 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da9e10-22da-4894-9292-da3d12194a0b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:08.214071 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129d312-086d-4bc2-aa53-e371b7927cf2 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:08.214629 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129d312-086d-4bc2-aa53-e371b7927cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:08.215078 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129d312-086d-4bc2-aa53-e371b7927cf2 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:08.217103 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 470/924] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:52:08.220457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:08.224627 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc41cf-bb9b-485a-bcc4-ba62d44eea9f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:08.225187 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc41cf-bb9b-485a-bcc4-ba62d44eea9f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:08.225930 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc41cf-bb9b-485a-bcc4-ba62d44eea9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:08.227594 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 455/925] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jul 21 15:52:08 2022] POST /identity/v3/endpoints => generated 347 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:52:08.275604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3bdc41cf-bb9b-485a-bcc4-ba62d44eea9f None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:08.277323 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3bdc41cf-bb9b-485a-bcc4-ba62d44eea9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:08.290110 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:08.372039 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 471/926] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 15:52:08.396014 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da9e10-22da-4894-9292-da3d12194a0b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:08.517965 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da9e10-22da-4894-9292-da3d12194a0b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:08.558260 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05138d67-95cb-4785-be19-13d66ab89ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:08.559493 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05138d67-95cb-4785-be19-13d66ab89ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:08.560079 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05138d67-95cb-4785-be19-13d66ab89ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:08.560705 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05138d67-95cb-4785-be19-13d66ab89ba3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:08.561609 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05138d67-95cb-4785-be19-13d66ab89ba3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:08.563274 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05138d67-95cb-4785-be19-13d66ab89ba3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:08.577050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-05138d67-95cb-4785-be19-13d66ab89ba3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 15:52:08.577050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:08.577050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 15:52:08.577050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 15:52:08.577050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 15:52:08.577050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 15:52:08.577050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 15:52:08.577050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:52:08.577050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 15:52:08.577050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:08.577050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 15:52:08.577050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:08.577050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:08.577050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:08.577050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:08.577050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:08.577050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:08.578202 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:08.578202 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:08.578202 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:08.578202 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:08.578202 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:08.578202 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:08.578202 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 15:52:08.578202 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 15:52:08.578202 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 15:52:08.578202 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 15:52:08.578202 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:08.578202 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:08.578202 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:08.578202 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:08.578202 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:08.578202 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:08.579085 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:08.579085 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:08.579085 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:08.579085 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:08.579085 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:08.579085 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:08.579085 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:08.579085 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:08.579085 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:08.579085 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:08.579085 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 15:52:08.579085 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:52:08.579085 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 15:52:08.579085 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:08.582074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 456/927] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jul 21 15:52:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:08.588613 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc41cf-bb9b-485a-bcc4-ba62d44eea9f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:08.716283 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc41cf-bb9b-485a-bcc4-ba62d44eea9f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:08.753507 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8614733-24f9-4146-a3a0-8ec759078076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:08.754596 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8614733-24f9-4146-a3a0-8ec759078076 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:08.755088 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8614733-24f9-4146-a3a0-8ec759078076 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:08.755598 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8614733-24f9-4146-a3a0-8ec759078076 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:08.756536 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8614733-24f9-4146-a3a0-8ec759078076 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:08.757052 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8614733-24f9-4146-a3a0-8ec759078076 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:08.758664 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8614733-24f9-4146-a3a0-8ec759078076 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:08.764522 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 472/928] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jul 21 15:52:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:08.770553 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05138d67-95cb-4785-be19-13d66ab89ba3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:08.842834 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:08.897890 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05138d67-95cb-4785-be19-13d66ab89ba3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:08.937417 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de41a5-8f9a-4ecb-b61f-0455705d40d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:08.938480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de41a5-8f9a-4ecb-b61f-0455705d40d9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:08.939001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de41a5-8f9a-4ecb-b61f-0455705d40d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:08.939482 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de41a5-8f9a-4ecb-b61f-0455705d40d9 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:08.940406 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2de41a5-8f9a-4ecb-b61f-0455705d40d9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:08.940920 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2de41a5-8f9a-4ecb-b61f-0455705d40d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:08.942533 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2de41a5-8f9a-4ecb-b61f-0455705d40d9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:08.948851 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 457/929] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jul 21 15:52:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:08.961680 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8614733-24f9-4146-a3a0-8ec759078076 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:08.961680 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:09.067308 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8614733-24f9-4146-a3a0-8ec759078076 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:09.104717 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfffff5-5312-4edd-bac1-7821e9c7e072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:09.105408 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfffff5-5312-4edd-bac1-7821e9c7e072 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:09.105732 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfffff5-5312-4edd-bac1-7821e9c7e072 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:09.106014 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfffff5-5312-4edd-bac1-7821e9c7e072 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:09.106833 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfffff5-5312-4edd-bac1-7821e9c7e072 None admin] RBAC: Adding query filter params (interface=public, service_id=fcc4a96e80af41b395ff893481f7e957, region_id=RegionOne) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:09.107150 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfffff5-5312-4edd-bac1-7821e9c7e072 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:09.108752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfffff5-5312-4edd-bac1-7821e9c7e072 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:09.117551 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 473/930] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Jul 21 15:52:08 2022] GET /identity/v3/endpoints?service_id=fcc4a96e80af41b395ff893481f7e957&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:09.123683 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de41a5-8f9a-4ecb-b61f-0455705d40d9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:09.203541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28357c-2d67-4c2e-8e94-7cc714b62b7c None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:09.203882 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28357c-2d67-4c2e-8e94-7cc714b62b7c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:09.204143 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28357c-2d67-4c2e-8e94-7cc714b62b7c None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:09.206195 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 474/931] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:52:09.213945 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe6efd-2d5d-4949-ad7c-14d1cbc234af None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:09.214224 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe6efd-2d5d-4949-ad7c-14d1cbc234af None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:09.214746 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe6efd-2d5d-4949-ad7c-14d1cbc234af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:09.243667 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-12fe6efd-2d5d-4949-ad7c-14d1cbc234af None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:09.245212 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-12fe6efd-2d5d-4949-ad7c-14d1cbc234af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:09.254637 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de41a5-8f9a-4ecb-b61f-0455705d40d9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:09.287544 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:09.298858 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498098e0-56d4-4e68-9a6d-8bce261c4e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:09.299549 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498098e0-56d4-4e68-9a6d-8bce261c4e1e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:09.299842 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498098e0-56d4-4e68-9a6d-8bce261c4e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:09.300135 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498098e0-56d4-4e68-9a6d-8bce261c4e1e None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:09.300837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498098e0-56d4-4e68-9a6d-8bce261c4e1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:09.303384 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498098e0-56d4-4e68-9a6d-8bce261c4e1e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:09.325320 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 458/932] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jul 21 15:52:09 2022] GET /identity/v3/services => generated 1497 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 15:52:09.331263 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:09.339042 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 475/933] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:52:09.374133 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498098e0-56d4-4e68-9a6d-8bce261c4e1e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:09.514060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498098e0-56d4-4e68-9a6d-8bce261c4e1e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:09.552047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfd971-0776-40ea-8823-10fb05a24fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:09.552715 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfd971-0776-40ea-8823-10fb05a24fba None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:09.552826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f7e7e-0c70-41f3-a7e4-069914c7c6df None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:09.553045 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfd971-0776-40ea-8823-10fb05a24fba None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:09.553155 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f7e7e-0c70-41f3-a7e4-069914c7c6df None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:09.553306 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfd971-0776-40ea-8823-10fb05a24fba None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:09.553407 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f7e7e-0c70-41f3-a7e4-069914c7c6df None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:09.554108 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccfd971-0776-40ea-8823-10fb05a24fba None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:09.555202 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 476/934] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:52:09.555600 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccfd971-0776-40ea-8823-10fb05a24fba None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:09.557115 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: WARNING keystone.server.flask.common [None req-fccfd971-0776-40ea-8823-10fb05a24fba None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 21 15:52:09.567234 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdb3f4-c0ab-4097-b7f4-e79dcc24c139 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:09.567541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdb3f4-c0ab-4097-b7f4-e79dcc24c139 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:09.567798 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdb3f4-c0ab-4097-b7f4-e79dcc24c139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:09.579078 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.sql.core [None req-fccfd971-0776-40ea-8823-10fb05a24fba None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 15:52:09.579078 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 21 15:52:09.579078 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [parameters: {'id': '64028bdd24a444d0a6aa943ee76938f2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 21 15:52:09.579078 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80238) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 15:52:09.587351 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-fccfd971-0776-40ea-8823-10fb05a24fba None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 64028bdd24a444d0a6aa943ee76938f2.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 64028bdd24a444d0a6aa943ee76938f2. Jul 21 15:52:09.587351 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:09.587351 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 15:52:09.587351 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 15:52:09.587351 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 15:52:09.587351 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 15:52:09.587351 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 15:52:09.587351 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 15:52:09.587351 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 15:52:09.587351 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application conn.query(q) Jul 21 15:52:09.587351 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 15:52:09.587351 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 15:52:09.587351 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 15:52:09.588464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application result.read() Jul 21 15:52:09.588464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 15:52:09.588464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 15:52:09.588464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 15:52:09.588464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 15:52:09.588464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 15:52:09.588464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 15:52:09.588464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 15:52:09.588464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 15:52:09.588464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 15:52:09.588464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:09.588464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 15:52:09.588464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:09.588464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:09.588464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 15:52:09.588464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 15:52:09.588464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 21 15:52:09.588464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 21 15:52:09.589636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 15:52:09.589636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application next(self.gen) Jul 21 15:52:09.589636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 15:52:09.589636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application yield resource Jul 21 15:52:09.589636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 15:52:09.589636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application next(self.gen) Jul 21 15:52:09.589636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 15:52:09.589636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 15:52:09.589636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 15:52:09.589636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 15:52:09.589636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 15:52:09.589636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise self.value Jul 21 15:52:09.589636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 15:52:09.589636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 15:52:09.589636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 15:52:09.589636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application session.commit() Jul 21 15:52:09.589636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 15:52:09.590601 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 15:52:09.590601 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 15:52:09.590601 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 15:52:09.590601 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 15:52:09.590601 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self.session.flush() Jul 21 15:52:09.590601 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 15:52:09.590601 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 15:52:09.590601 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 15:52:09.590601 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 15:52:09.590601 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 15:52:09.590601 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application compat.raise_( Jul 21 15:52:09.590601 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 15:52:09.590601 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception Jul 21 15:52:09.590601 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 15:52:09.590601 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 15:52:09.590601 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 15:52:09.590601 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 15:52:09.591631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 15:52:09.591631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 15:52:09.591631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 15:52:09.591631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 15:52:09.591631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 21 15:52:09.591631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 21 15:52:09.591631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 15:52:09.591631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 15:52:09.591631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 15:52:09.591631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 15:52:09.591631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 15:52:09.591631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 15:52:09.591631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 15:52:09.591631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 15:52:09.591631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 15:52:09.594636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 15:52:09.594636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 15:52:09.594636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception Jul 21 15:52:09.594636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 15:52:09.594636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 15:52:09.594636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 15:52:09.594636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 15:52:09.594636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 15:52:09.594636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 15:52:09.594636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 15:52:09.594636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application conn.query(q) Jul 21 15:52:09.594636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 15:52:09.594636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 15:52:09.594636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 15:52:09.594636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application result.read() Jul 21 15:52:09.594636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 15:52:09.594636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 15:52:09.596941 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 15:52:09.596941 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 15:52:09.596941 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 15:52:09.596941 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 15:52:09.596941 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 15:52:09.596941 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 15:52:09.596941 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 15:52:09.596941 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 21 15:52:09.596941 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [parameters: {'id': '64028bdd24a444d0a6aa943ee76938f2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 21 15:52:09.596941 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 15:52:09.596941 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:09.596941 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 15:52:09.596941 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:09.596941 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:09.596941 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 21 15:52:09.600457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 21 15:52:09.600457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 15:52:09.600457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 15:52:09.600457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 21 15:52:09.600457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:09.600457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 15:52:09.600457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:09.600457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:09.600457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:09.600457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:09.600457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:09.600457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:09.600457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:09.600457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:09.600457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:09.600457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:09.600457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:09.602948 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:09.602948 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 21 15:52:09.602948 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 21 15:52:09.602948 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:09.602948 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:09.602948 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 21 15:52:09.602948 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 21 15:52:09.602948 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 64028bdd24a444d0a6aa943ee76938f2. Jul 21 15:52:09.602948 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:09.602948 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 459/935] 10.222.0.52 () {60 vars in 1140 bytes} [Thu Jul 21 15:52:09 2022] POST /identity/v3/projects => generated 262 bytes in 215 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:09.602948 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfd971-0776-40ea-8823-10fb05a24fba None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:09.618675 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-efcdb3f4-c0ab-4097-b7f4-e79dcc24c139 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:09.621157 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-efcdb3f4-c0ab-4097-b7f4-e79dcc24c139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:09.662209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:09.737566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 477/936] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:52:09.759367 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfd971-0776-40ea-8823-10fb05a24fba None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:09.762390 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcdb3f4-c0ab-4097-b7f4-e79dcc24c139 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:09.806768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac926a-66ed-4476-81f2-8bb76c1271aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:09.808068 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac926a-66ed-4476-81f2-8bb76c1271aa None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:09.808616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac926a-66ed-4476-81f2-8bb76c1271aa None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:09.809122 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac926a-66ed-4476-81f2-8bb76c1271aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:09.817713 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bac926a-66ed-4476-81f2-8bb76c1271aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:09.819877 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bac926a-66ed-4476-81f2-8bb76c1271aa None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:09.836413 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-5bac926a-66ed-4476-81f2-8bb76c1271aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:52:09.836413 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:09.836413 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:09.836413 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:09.836413 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:09.836413 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:09.836413 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:09.836413 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:09.836413 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:09.836413 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:09.836413 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:09.836413 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:09.836413 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:52:09.836413 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:52:09.836413 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:52:09.837408 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:52:09.837408 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:09.837408 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:09.837408 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:09.837408 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:09.837408 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:09.837408 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:09.837408 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:09.837408 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:09.837408 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:09.837408 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:09.837408 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:09.837408 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:09.837408 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:09.837408 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:09.837408 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:09.837408 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:09.839278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:52:09.839278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:52:09.839278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:52:09.839278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:52:09.839278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:52:09.839278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:52:09.839278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:52:09.839278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:09.846704 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 460/937] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:52:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:52:09.848216 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac926a-66ed-4476-81f2-8bb76c1271aa None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:09.886864 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcdb3f4-c0ab-4097-b7f4-e79dcc24c139 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:09.924102 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:09.934457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f441f5-04cf-4119-9918-05aa6f54e55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:09.938507 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f441f5-04cf-4119-9918-05aa6f54e55f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:09.939060 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f441f5-04cf-4119-9918-05aa6f54e55f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:09.939492 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f441f5-04cf-4119-9918-05aa6f54e55f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:09.961769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f441f5-04cf-4119-9918-05aa6f54e55f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:52:09.970674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-08f441f5-04cf-4119-9918-05aa6f54e55f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 15:52:09.970674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:09.970674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:09.970674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:09.970674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:09.970674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:09.970674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:09.970674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:09.970674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:09.970674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:09.970674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:09.970674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:09.970674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:52:09.970674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:52:09.970674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:52:09.970674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:52:09.971911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:52:09.971911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:52:09.971911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:52:09.971911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:52:09.971911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:09.971911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:09.971911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:09.971911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:09.971911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:09.971911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:09.971911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:09.971911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:09.971911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:09.971911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:09.971911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:09.971911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:09.973279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:09.973279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:09.973279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:09.973279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:09.973279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:52:09.973279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:52:09.973279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:52:09.973279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:52:09.973279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:52:09.973279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:52:09.973279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 15:52:09.973279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:09.980273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 478/938] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Jul 21 15:52:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:09.981327 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f441f5-04cf-4119-9918-05aa6f54e55f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:09.988240 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac926a-66ed-4476-81f2-8bb76c1271aa None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:09.998947 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:10.030592 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85053e-d91d-4dc1-8938-e4cd1928bb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:10.031478 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85053e-d91d-4dc1-8938-e4cd1928bb3f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:10.032547 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85053e-d91d-4dc1-8938-e4cd1928bb3f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:10.032547 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85053e-d91d-4dc1-8938-e4cd1928bb3f None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:10.033291 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85053e-d91d-4dc1-8938-e4cd1928bb3f None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:10.033746 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85053e-d91d-4dc1-8938-e4cd1928bb3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:10.035295 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85053e-d91d-4dc1-8938-e4cd1928bb3f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:10.062006 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 461/939] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 21 15:52:09 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:10.113147 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f441f5-04cf-4119-9918-05aa6f54e55f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:10.155687 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bfc0f7-99ca-4254-9e58-92bed6fdd059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:10.156646 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bfc0f7-99ca-4254-9e58-92bed6fdd059 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:10.157118 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bfc0f7-99ca-4254-9e58-92bed6fdd059 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:10.157548 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bfc0f7-99ca-4254-9e58-92bed6fdd059 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:10.158500 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bfc0f7-99ca-4254-9e58-92bed6fdd059 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:10.158968 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bfc0f7-99ca-4254-9e58-92bed6fdd059 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:10.160569 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bfc0f7-99ca-4254-9e58-92bed6fdd059 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:10.171613 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 479/940] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Jul 21 15:52:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:10.177698 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85053e-d91d-4dc1-8938-e4cd1928bb3f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:10.323358 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85053e-d91d-4dc1-8938-e4cd1928bb3f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:10.365377 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993e203-5bcb-459e-8826-ea869a471619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:10.366452 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993e203-5bcb-459e-8826-ea869a471619 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:10.366890 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993e203-5bcb-459e-8826-ea869a471619 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:10.367315 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993e203-5bcb-459e-8826-ea869a471619 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:10.382155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a993e203-5bcb-459e-8826-ea869a471619 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:10.385205 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a993e203-5bcb-459e-8826-ea869a471619 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:10.404616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-a993e203-5bcb-459e-8826-ea869a471619 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:52:10.404616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:10.404616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:10.404616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:10.404616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:10.404616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:10.404616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:10.404616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:10.404616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:10.404616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:10.404616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:10.404616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:10.404616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:52:10.404616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:52:10.404616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:52:10.405788 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:52:10.405788 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:10.405788 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:10.405788 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:10.405788 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:10.405788 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:10.405788 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:10.405788 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:10.405788 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:10.405788 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:10.405788 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:10.405788 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:10.405788 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:10.405788 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:10.405788 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:10.405788 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:10.405788 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:10.406945 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:52:10.406945 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:52:10.406945 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:52:10.406945 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:10.408832 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 462/941] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:52:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:10.417150 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bfc0f7-99ca-4254-9e58-92bed6fdd059 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:10.469707 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:10.528702 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bfc0f7-99ca-4254-9e58-92bed6fdd059 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:10.567126 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f94ecb-57c4-4369-aca0-31c0dfe13525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:10.567881 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f94ecb-57c4-4369-aca0-31c0dfe13525 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:10.568171 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f94ecb-57c4-4369-aca0-31c0dfe13525 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:10.568451 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f94ecb-57c4-4369-aca0-31c0dfe13525 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:10.569199 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f94ecb-57c4-4369-aca0-31c0dfe13525 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:10.569512 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f94ecb-57c4-4369-aca0-31c0dfe13525 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:10.571459 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ecc6e8-c262-4b36-9f59-769c44ca21f2 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:10.571747 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ecc6e8-c262-4b36-9f59-769c44ca21f2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:10.572025 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ecc6e8-c262-4b36-9f59-769c44ca21f2 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:10.573632 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 463/942] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 15:52:10.574836 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f94ecb-57c4-4369-aca0-31c0dfe13525 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:10.582730 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891d1b0-c5ac-4f2c-b23a-6965f6ca15c1 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:10.582730 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891d1b0-c5ac-4f2c-b23a-6965f6ca15c1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:10.582730 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891d1b0-c5ac-4f2c-b23a-6965f6ca15c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:10.598918 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 480/943] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:52:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:10.606069 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f94ecb-57c4-4369-aca0-31c0dfe13525 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:10.606257 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b891d1b0-c5ac-4f2c-b23a-6965f6ca15c1 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:10.607911 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b891d1b0-c5ac-4f2c-b23a-6965f6ca15c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:10.625884 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:10.698884 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 464/944] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:52:10.723547 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891d1b0-c5ac-4f2c-b23a-6965f6ca15c1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:10.747261 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f94ecb-57c4-4369-aca0-31c0dfe13525 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:10.764478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:10.804952 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645821d-66a3-453c-bcaa-5dddbf36e77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:10.805855 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645821d-66a3-453c-bcaa-5dddbf36e77c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:10.806143 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645821d-66a3-453c-bcaa-5dddbf36e77c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:10.806423 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645821d-66a3-453c-bcaa-5dddbf36e77c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:10.817061 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0645821d-66a3-453c-bcaa-5dddbf36e77c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:10.819332 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0645821d-66a3-453c-bcaa-5dddbf36e77c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:10.838759 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-0645821d-66a3-453c-bcaa-5dddbf36e77c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 15:52:10.838759 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:10.838759 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:10.838759 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:10.838759 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:10.838759 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:10.838759 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:10.838759 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:10.838759 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:10.838759 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:10.838759 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:10.838759 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:10.838759 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:52:10.838759 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:52:10.838759 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:52:10.838759 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:52:10.839797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:10.839797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:10.839797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:52:10.839797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:52:10.839797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:52:10.839797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:52:10.839797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:10.839797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:10.839797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:10.839797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:10.839797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:10.839797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:10.839797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:10.839797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:10.839797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:10.839797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:10.839797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:10.839797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:10.840770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:10.840770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:10.840770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:52:10.840770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:52:10.840770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:52:10.840770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:52:10.840770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:52:10.840770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:52:10.840770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 15:52:10.840770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:10.840770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 481/945] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jul 21 15:52:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:52:10.849086 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645821d-66a3-453c-bcaa-5dddbf36e77c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:10.859731 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891d1b0-c5ac-4f2c-b23a-6965f6ca15c1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:10.910374 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468488c-b5e4-48ad-80fd-a5a25e7b2d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:10.911966 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468488c-b5e4-48ad-80fd-a5a25e7b2d92 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:10.912703 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468488c-b5e4-48ad-80fd-a5a25e7b2d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:10.913230 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468488c-b5e4-48ad-80fd-a5a25e7b2d92 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:10.914763 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0468488c-b5e4-48ad-80fd-a5a25e7b2d92 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:10.919321 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0468488c-b5e4-48ad-80fd-a5a25e7b2d92 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:10.946782 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-0468488c-b5e4-48ad-80fd-a5a25e7b2d92 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 15:52:10.946782 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:10.946782 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 15:52:10.946782 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 15:52:10.946782 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 15:52:10.946782 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 15:52:10.946782 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 15:52:10.946782 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:52:10.946782 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 15:52:10.946782 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:10.946782 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 15:52:10.946782 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:10.946782 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:10.946782 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:10.946782 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:10.946782 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:10.946782 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:10.948648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:10.948648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:10.948648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:10.948648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:10.948648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:10.948648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:10.948648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 15:52:10.948648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 15:52:10.948648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 15:52:10.948648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 15:52:10.948648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:10.948648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:10.948648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:10.948648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:10.948648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:10.948648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:10.949808 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:10.949808 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:10.949808 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:10.949808 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:10.949808 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:10.949808 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:10.949808 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:10.949808 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:10.949808 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:10.949808 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:10.949808 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 15:52:10.949808 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:52:10.949808 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 15:52:10.949808 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:10.952095 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 465/946] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:52:10 2022] GET /identity/v3/services/compute => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:10.961767 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468488c-b5e4-48ad-80fd-a5a25e7b2d92 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:10.977049 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:10.992520 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645821d-66a3-453c-bcaa-5dddbf36e77c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:11.048774 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb7f383-5591-436e-9c1e-e01ee612ab69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:11.049467 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7f383-5591-436e-9c1e-e01ee612ab69 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:11.049783 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7f383-5591-436e-9c1e-e01ee612ab69 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:11.050067 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7f383-5591-436e-9c1e-e01ee612ab69 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:11.050930 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb7f383-5591-436e-9c1e-e01ee612ab69 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:11.051249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb7f383-5591-436e-9c1e-e01ee612ab69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:11.052716 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb7f383-5591-436e-9c1e-e01ee612ab69 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:11.078167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 482/947] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Jul 21 15:52:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:11.083594 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb7f383-5591-436e-9c1e-e01ee612ab69 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:11.095281 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468488c-b5e4-48ad-80fd-a5a25e7b2d92 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:11.139972 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:11.159425 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e131f2-97dd-443b-a039-a259830f7bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:11.160112 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e131f2-97dd-443b-a039-a259830f7bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:11.160458 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e131f2-97dd-443b-a039-a259830f7bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:11.160685 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e131f2-97dd-443b-a039-a259830f7bc4 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:11.161754 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e131f2-97dd-443b-a039-a259830f7bc4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:11.161875 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e131f2-97dd-443b-a039-a259830f7bc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:11.163293 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e131f2-97dd-443b-a039-a259830f7bc4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:11.174128 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 466/948] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 21 15:52:10 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:52:11.176867 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e131f2-97dd-443b-a039-a259830f7bc4 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:11.242157 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb7f383-5591-436e-9c1e-e01ee612ab69 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:11.278584 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:11.291899 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910b681-9d74-40e7-8bea-51350fe4484b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:11.292899 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910b681-9d74-40e7-8bea-51350fe4484b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:11.293346 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910b681-9d74-40e7-8bea-51350fe4484b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:11.295271 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910b681-9d74-40e7-8bea-51350fe4484b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:11.312568 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e131f2-97dd-443b-a039-a259830f7bc4 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:11.312772 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5910b681-9d74-40e7-8bea-51350fe4484b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:11.314242 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5910b681-9d74-40e7-8bea-51350fe4484b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:11.343090 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-5910b681-9d74-40e7-8bea-51350fe4484b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:52:11.343090 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:11.343090 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:11.343090 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:11.343090 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:11.343090 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:11.343090 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:11.343090 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:11.343090 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:11.343090 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:11.343090 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:11.343090 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:11.343090 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:52:11.343090 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:52:11.343090 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:52:11.344225 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:52:11.344225 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:11.344225 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:11.344225 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:11.344225 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:11.344225 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:11.344225 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:11.344225 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:11.344225 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:11.344225 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:11.344225 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:11.344225 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:11.344225 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:11.344225 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:11.344225 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:11.344225 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:11.344225 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:11.345410 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:52:11.345410 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:52:11.345410 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:52:11.345410 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:11.345410 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 483/949] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:52:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:11.351643 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910b681-9d74-40e7-8bea-51350fe4484b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:11.364655 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b63a1-3193-4917-bd4b-a45fceb50a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:11.365447 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b63a1-3193-4917-bd4b-a45fceb50a45 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:11.365789 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b63a1-3193-4917-bd4b-a45fceb50a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:11.366076 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b63a1-3193-4917-bd4b-a45fceb50a45 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:11.366916 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392b63a1-3193-4917-bd4b-a45fceb50a45 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:11.367340 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392b63a1-3193-4917-bd4b-a45fceb50a45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:11.368873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392b63a1-3193-4917-bd4b-a45fceb50a45 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:11.375128 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 467/950] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 21 15:52:11 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:11.469741 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:11.496281 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910b681-9d74-40e7-8bea-51350fe4484b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:11.540218 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17bfdbd-6aeb-492e-ae2b-8a8e7e7a4f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:11.541236 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17bfdbd-6aeb-492e-ae2b-8a8e7e7a4f59 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:11.541738 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17bfdbd-6aeb-492e-ae2b-8a8e7e7a4f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:11.542197 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17bfdbd-6aeb-492e-ae2b-8a8e7e7a4f59 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:11.543471 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17bfdbd-6aeb-492e-ae2b-8a8e7e7a4f59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:11.543995 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17bfdbd-6aeb-492e-ae2b-8a8e7e7a4f59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:11.546934 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17bfdbd-6aeb-492e-ae2b-8a8e7e7a4f59 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:11.563475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 484/951] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:52:11.570243 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b63a1-3193-4917-bd4b-a45fceb50a45 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:11.581720 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f0092-3fde-4cbc-99d9-f98e4ac129f7 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:11.582019 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f0092-3fde-4cbc-99d9-f98e4ac129f7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:11.582309 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f0092-3fde-4cbc-99d9-f98e4ac129f7 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:11.584570 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 485/952] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:52:11.590167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485eec85-c681-430c-a326-672c5a445573 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:11.590464 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485eec85-c681-430c-a326-672c5a445573 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:11.590747 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485eec85-c681-430c-a326-672c5a445573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:11.609843 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-485eec85-c681-430c-a326-672c5a445573 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:11.611075 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-485eec85-c681-430c-a326-672c5a445573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:11.694084 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b63a1-3193-4917-bd4b-a45fceb50a45 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:11.709730 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 486/953] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 15:52:11.734617 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485eec85-c681-430c-a326-672c5a445573 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:11.749260 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3b1ab-4c52-4001-a20d-65d1545d05a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:11.750865 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3b1ab-4c52-4001-a20d-65d1545d05a9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:11.752073 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3b1ab-4c52-4001-a20d-65d1545d05a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:11.752861 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3b1ab-4c52-4001-a20d-65d1545d05a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:11.763276 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d3b1ab-4c52-4001-a20d-65d1545d05a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:11.765767 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d3b1ab-4c52-4001-a20d-65d1545d05a9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:11.770478 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:11.787832 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-a6d3b1ab-4c52-4001-a20d-65d1545d05a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:52:11.787832 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:11.787832 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:11.787832 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:11.787832 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:11.787832 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:11.787832 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:11.787832 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:11.787832 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:11.787832 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:11.787832 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:11.787832 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:11.787832 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:52:11.787832 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:52:11.787832 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:52:11.789750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:52:11.789750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:11.789750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:11.789750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:11.789750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:11.789750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:11.789750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:11.789750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:11.789750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:11.789750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:11.789750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:11.789750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:11.789750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:11.789750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:11.789750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:11.789750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:11.789750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:11.791590 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:52:11.791590 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:52:11.791590 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:52:11.791590 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:52:11.791590 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:52:11.791590 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:52:11.791590 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:52:11.791590 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:11.797778 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 468/954] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:52:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:11.801594 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3b1ab-4c52-4001-a20d-65d1545d05a9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:11.836996 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:11.882851 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485eec85-c681-430c-a326-672c5a445573 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:11.934968 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d96fca-ec9a-4831-be7c-91f7bb540705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:11.936766 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d96fca-ec9a-4831-be7c-91f7bb540705 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:11.937299 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d96fca-ec9a-4831-be7c-91f7bb540705 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:11.937816 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d96fca-ec9a-4831-be7c-91f7bb540705 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:11.939009 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d96fca-ec9a-4831-be7c-91f7bb540705 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:11.941412 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d96fca-ec9a-4831-be7c-91f7bb540705 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:11.947455 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3b1ab-4c52-4001-a20d-65d1545d05a9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:11.958118 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-09d96fca-ec9a-4831-be7c-91f7bb540705 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 15:52:11.958118 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:11.958118 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 15:52:11.958118 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 15:52:11.958118 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 15:52:11.958118 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 15:52:11.958118 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 15:52:11.958118 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:52:11.958118 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 15:52:11.958118 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:11.958118 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 15:52:11.958118 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:11.958118 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:11.958118 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:11.958118 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:11.958118 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:11.958118 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:11.959292 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:11.959292 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:11.959292 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:11.959292 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:11.959292 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:11.959292 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:11.959292 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 15:52:11.959292 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 15:52:11.959292 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 15:52:11.959292 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 15:52:11.959292 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:11.959292 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:11.959292 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:11.959292 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:11.959292 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:11.959292 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:11.960252 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:11.960252 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:11.960252 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:11.960252 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:11.960252 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:11.960252 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:11.960252 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:11.960252 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:11.960252 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:11.960252 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:11.960252 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 15:52:11.960252 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:52:11.960252 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 15:52:11.960252 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:11.968917 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 487/955] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jul 21 15:52:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:52:11.969865 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d96fca-ec9a-4831-be7c-91f7bb540705 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:11.996359 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2840a-5f53-4b44-946d-1717bf4bed36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:11.997373 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2840a-5f53-4b44-946d-1717bf4bed36 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:11.998418 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2840a-5f53-4b44-946d-1717bf4bed36 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:11.998937 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2840a-5f53-4b44-946d-1717bf4bed36 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:11.999986 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2840a-5f53-4b44-946d-1717bf4bed36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:12.000494 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2840a-5f53-4b44-946d-1717bf4bed36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:12.002235 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2840a-5f53-4b44-946d-1717bf4bed36 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:12.020512 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 469/956] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:52:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:12.021827 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2840a-5f53-4b44-946d-1717bf4bed36 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:12.079765 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d96fca-ec9a-4831-be7c-91f7bb540705 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:12.087728 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:12.119191 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360fcc23-7a55-4f43-9f85-959e15724efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:12.119858 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360fcc23-7a55-4f43-9f85-959e15724efb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:12.120142 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360fcc23-7a55-4f43-9f85-959e15724efb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:12.120417 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360fcc23-7a55-4f43-9f85-959e15724efb None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:12.121304 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360fcc23-7a55-4f43-9f85-959e15724efb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:12.121539 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360fcc23-7a55-4f43-9f85-959e15724efb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:12.123003 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360fcc23-7a55-4f43-9f85-959e15724efb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:12.124103 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:12.133886 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 488/957] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jul 21 15:52:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:12.135824 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360fcc23-7a55-4f43-9f85-959e15724efb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:12.141071 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2840a-5f53-4b44-946d-1717bf4bed36 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:12.187634 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd26bc-f69c-4392-9aa7-60c704ae3333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:12.188915 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd26bc-f69c-4392-9aa7-60c704ae3333 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:12.189420 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd26bc-f69c-4392-9aa7-60c704ae3333 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:12.189972 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd26bc-f69c-4392-9aa7-60c704ae3333 None admin] PATH_INFO: `/v3/projects/bb1ac00308714c39a79aae09fbb73021/users/6edbad98ed0f4823b9955f9f0e270149/roles/abd0095f0bf64a0c8e694dbf50d6fd9a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:12.192605 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd26bc-f69c-4392-9aa7-60c704ae3333 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb1ac00308714c39a79aae09fbb73021, user_id=6edbad98ed0f4823b9955f9f0e270149, role_id=abd0095f0bf64a0c8e694dbf50d6fd9a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:12.194434 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd26bc-f69c-4392-9aa7-60c704ae3333 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:12.221882 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 470/958] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 15:52:12 2022] PUT /identity/v3/projects/bb1ac00308714c39a79aae09fbb73021/users/6edbad98ed0f4823b9955f9f0e270149/roles/abd0095f0bf64a0c8e694dbf50d6fd9a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 15:52:12.297306 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360fcc23-7a55-4f43-9f85-959e15724efb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:12.338308 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b8833-2d82-4e67-9d1c-44aaa43faf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:12.339045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b8833-2d82-4e67-9d1c-44aaa43faf69 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:12.339336 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b8833-2d82-4e67-9d1c-44aaa43faf69 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:12.339615 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b8833-2d82-4e67-9d1c-44aaa43faf69 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:12.340350 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b8833-2d82-4e67-9d1c-44aaa43faf69 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:12.340663 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b8833-2d82-4e67-9d1c-44aaa43faf69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:12.342912 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b8833-2d82-4e67-9d1c-44aaa43faf69 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:12.344833 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee37a5-2060-484f-8feb-3f9beccbf2ab None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:12.345286 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee37a5-2060-484f-8feb-3f9beccbf2ab None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:12.345389 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee37a5-2060-484f-8feb-3f9beccbf2ab None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:12.347245 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 471/959] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 15:52:12.349719 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 489/960] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jul 21 15:52:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:12.353318 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c094eca5-b005-4395-b58e-5420a4ba5286 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:12.353635 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c094eca5-b005-4395-b58e-5420a4ba5286 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:12.353926 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c094eca5-b005-4395-b58e-5420a4ba5286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:12.355297 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b8833-2d82-4e67-9d1c-44aaa43faf69 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:12.372616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c094eca5-b005-4395-b58e-5420a4ba5286 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:12.373965 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c094eca5-b005-4395-b58e-5420a4ba5286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:12.455767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:12.480305 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 472/961] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 15:52:12.505560 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b8833-2d82-4e67-9d1c-44aaa43faf69 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:12.523428 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c094eca5-b005-4395-b58e-5420a4ba5286 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:12.553622 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3ce4f-35d2-4899-927f-2958854ac247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:12.554548 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3ce4f-35d2-4899-927f-2958854ac247 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:12.555009 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3ce4f-35d2-4899-927f-2958854ac247 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:12.555468 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3ce4f-35d2-4899-927f-2958854ac247 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:12.556773 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3ce4f-35d2-4899-927f-2958854ac247 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:12.559189 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3ce4f-35d2-4899-927f-2958854ac247 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:12.592984 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 490/962] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jul 21 15:52:12 2022] POST /identity/v3/endpoints => generated 349 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 15:52:12.629642 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:12.694802 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c094eca5-b005-4395-b58e-5420a4ba5286 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:12.748938 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54afa4-26eb-429b-a8ef-ff4c7a498ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:12.749721 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54afa4-26eb-429b-a8ef-ff4c7a498ab0 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:12.750006 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54afa4-26eb-429b-a8ef-ff4c7a498ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:12.750332 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54afa4-26eb-429b-a8ef-ff4c7a498ab0 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:12.751065 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54afa4-26eb-429b-a8ef-ff4c7a498ab0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:12.752553 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54afa4-26eb-429b-a8ef-ff4c7a498ab0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:12.767040 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.sql.core [None req-fe54afa4-26eb-429b-a8ef-ff4c7a498ab0 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 15:52:12.767040 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 21 15:52:12.767040 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [parameters: {'user_id': 'f7bf2e7455884b07bc8b82c06d926c43', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 21 15:52:12.767040 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80238) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 15:52:12.774426 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-fe54afa4-26eb-429b-a8ef-ff4c7a498ab0 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 21 15:52:12.774426 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:12.774426 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 15:52:12.774426 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 15:52:12.774426 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 15:52:12.774426 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 15:52:12.774426 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 15:52:12.774426 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 15:52:12.774426 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 15:52:12.774426 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application conn.query(q) Jul 21 15:52:12.774426 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 15:52:12.774426 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 15:52:12.774426 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 15:52:12.774426 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application result.read() Jul 21 15:52:12.774426 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 15:52:12.775534 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 15:52:12.775534 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 15:52:12.775534 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 15:52:12.775534 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 15:52:12.775534 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 15:52:12.775534 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 15:52:12.775534 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 15:52:12.775534 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 15:52:12.775534 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:12.775534 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 15:52:12.775534 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:12.775534 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:12.775534 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 15:52:12.775534 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 15:52:12.775534 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 21 15:52:12.775534 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 21 15:52:12.775534 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 15:52:12.775534 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application next(self.gen) Jul 21 15:52:12.776783 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 15:52:12.776783 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application yield resource Jul 21 15:52:12.776783 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 15:52:12.776783 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application next(self.gen) Jul 21 15:52:12.776783 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 15:52:12.776783 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 15:52:12.776783 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 15:52:12.776783 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 15:52:12.776783 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 15:52:12.776783 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise self.value Jul 21 15:52:12.776783 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 15:52:12.776783 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 15:52:12.776783 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 15:52:12.776783 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application session.commit() Jul 21 15:52:12.776783 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 15:52:12.776783 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 15:52:12.776783 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 15:52:12.778142 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 15:52:12.778142 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 15:52:12.778142 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self.session.flush() Jul 21 15:52:12.778142 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 15:52:12.778142 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 15:52:12.778142 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 15:52:12.778142 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 15:52:12.778142 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 15:52:12.778142 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application compat.raise_( Jul 21 15:52:12.778142 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 15:52:12.778142 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception Jul 21 15:52:12.778142 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 15:52:12.778142 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 15:52:12.778142 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 15:52:12.778142 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 15:52:12.778142 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 15:52:12.778142 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 15:52:12.779883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 15:52:12.779883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 15:52:12.779883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 21 15:52:12.779883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 21 15:52:12.779883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 15:52:12.779883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 15:52:12.779883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 15:52:12.779883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 15:52:12.779883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 15:52:12.779883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 15:52:12.779883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 15:52:12.779883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 15:52:12.779883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 15:52:12.779883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 15:52:12.779883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 15:52:12.779883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception Jul 21 15:52:12.780985 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 15:52:12.780985 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 15:52:12.780985 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 15:52:12.780985 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 15:52:12.780985 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 15:52:12.780985 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 15:52:12.780985 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 15:52:12.780985 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application conn.query(q) Jul 21 15:52:12.780985 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 15:52:12.780985 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 15:52:12.780985 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 15:52:12.780985 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application result.read() Jul 21 15:52:12.780985 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 15:52:12.780985 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 15:52:12.780985 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 15:52:12.780985 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 15:52:12.782838 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 15:52:12.782838 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 15:52:12.782838 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 15:52:12.782838 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 15:52:12.782838 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 15:52:12.782838 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 21 15:52:12.782838 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [parameters: {'user_id': 'f7bf2e7455884b07bc8b82c06d926c43', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 21 15:52:12.782838 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 15:52:12.782838 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:12.782838 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 15:52:12.782838 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:12.782838 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:12.782838 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:12.782838 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:12.782838 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:12.783988 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:12.783988 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:12.783988 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:12.783988 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:12.783988 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:12.783988 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:12.783988 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:12.783988 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 21 15:52:12.783988 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 21 15:52:12.783988 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:12.783988 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:12.783988 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:52:12.783988 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:52:12.783988 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:52:12.783988 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:52:12.783988 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 21 15:52:12.783988 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 21 15:52:12.785154 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 21 15:52:12.785154 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 21 15:52:12.785154 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 15:52:12.785154 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 15:52:12.785154 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 21 15:52:12.785154 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:12.785154 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 473/963] 10.222.0.52 () {60 vars in 1131 bytes} [Thu Jul 21 15:52:12 2022] POST /identity/v3/users => generated 160 bytes in 253 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:52:12.785697 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3ce4f-35d2-4899-927f-2958854ac247 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:12.923175 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3ce4f-35d2-4899-927f-2958854ac247 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:12.942182 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:12.964622 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5da6673-b6d6-41af-92bb-19908efc3404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:12.965712 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da6673-b6d6-41af-92bb-19908efc3404 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:12.966142 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da6673-b6d6-41af-92bb-19908efc3404 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:12.966757 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da6673-b6d6-41af-92bb-19908efc3404 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:12.976697 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5da6673-b6d6-41af-92bb-19908efc3404 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:12.978453 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5da6673-b6d6-41af-92bb-19908efc3404 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:12.994184 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-a5da6673-b6d6-41af-92bb-19908efc3404 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 15:52:12.994184 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:12.994184 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:12.994184 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:12.994184 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:12.994184 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:12.994184 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:12.994184 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:12.994184 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:12.994184 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:12.994184 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:12.994184 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:12.994184 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:52:12.994184 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:52:12.994184 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:52:12.994184 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:52:12.995440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:12.995440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:12.995440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:52:12.995440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:52:12.995440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:52:12.995440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:52:12.995440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:12.995440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:12.995440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:12.995440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:12.995440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:12.995440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:12.995440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:12.995440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:12.995440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:12.995440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:12.995440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:12.995440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:12.996648 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:12.996648 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:12.996648 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:52:12.996648 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:52:12.996648 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:52:12.996648 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:52:12.996648 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:52:12.996648 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:52:12.996648 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 15:52:12.996648 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:12.999115 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 491/964] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:52:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:13.005747 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54afa4-26eb-429b-a8ef-ff4c7a498ab0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:13.124520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54afa4-26eb-429b-a8ef-ff4c7a498ab0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:13.134405 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:13.166625 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a5dc4d-9788-4f30-8df5-aa0514d79f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:13.167309 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a5dc4d-9788-4f30-8df5-aa0514d79f95 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:13.167597 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a5dc4d-9788-4f30-8df5-aa0514d79f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:13.167874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a5dc4d-9788-4f30-8df5-aa0514d79f95 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:13.168705 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a5dc4d-9788-4f30-8df5-aa0514d79f95 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:13.169019 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a5dc4d-9788-4f30-8df5-aa0514d79f95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:13.170720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a5dc4d-9788-4f30-8df5-aa0514d79f95 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:13.218310 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 474/965] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 15:52:13 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:13.608658 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe1bb3-5952-4356-b3bc-c732a19dc4c8 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:13.609224 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe1bb3-5952-4356-b3bc-c732a19dc4c8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:13.609224 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe1bb3-5952-4356-b3bc-c732a19dc4c8 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:13.610489 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 492/966] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:52:13.620980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c3b90d-5067-41dd-bc2e-2148f1e08011 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:13.621488 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c3b90d-5067-41dd-bc2e-2148f1e08011 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:13.621719 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c3b90d-5067-41dd-bc2e-2148f1e08011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:13.639826 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-53c3b90d-5067-41dd-bc2e-2148f1e08011 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:13.641391 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-53c3b90d-5067-41dd-bc2e-2148f1e08011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:13.725445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 475/967] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:52:13.748066 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbe1bb3-5952-4356-b3bc-c732a19dc4c8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:13.881899 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbe1bb3-5952-4356-b3bc-c732a19dc4c8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:13.927214 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bffdfc7-a7ad-4c73-a609-290d5f1676ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:13.927894 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffdfc7-a7ad-4c73-a609-290d5f1676ca None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:13.928180 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffdfc7-a7ad-4c73-a609-290d5f1676ca None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:13.928462 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffdfc7-a7ad-4c73-a609-290d5f1676ca None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:13.929197 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bffdfc7-a7ad-4c73-a609-290d5f1676ca None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:13.930634 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bffdfc7-a7ad-4c73-a609-290d5f1676ca None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:13.948428 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 493/968] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 21 15:52:13 2022] POST /identity/v3/services => generated 242 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:52:14.506950 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f710fa-b7d9-454a-ab75-350c5d96bf84 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:14.507253 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f710fa-b7d9-454a-ab75-350c5d96bf84 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:14.507436 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f710fa-b7d9-454a-ab75-350c5d96bf84 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:14.508839 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 476/969] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:52:14.514489 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af311b-cde5-45ab-9eca-48f3a750ab34 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:14.514784 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af311b-cde5-45ab-9eca-48f3a750ab34 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:14.515070 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af311b-cde5-45ab-9eca-48f3a750ab34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:14.534029 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a5af311b-cde5-45ab-9eca-48f3a750ab34 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:14.535294 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a5af311b-cde5-45ab-9eca-48f3a750ab34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:14.572678 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:14.624952 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 494/970] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:14 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:52:14.648347 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f710fa-b7d9-454a-ab75-350c5d96bf84 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:14.758307 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:14.765191 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc193c-21c3-4da4-84af-e8eb390d21db None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:14.765499 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc193c-21c3-4da4-84af-e8eb390d21db None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:14.765809 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc193c-21c3-4da4-84af-e8eb390d21db None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:14.767278 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 495/971] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 15:52:14.767394 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f710fa-b7d9-454a-ab75-350c5d96bf84 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:14.773938 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a31f16-e90c-425a-92e6-d6b3e26c0ec6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:14.774217 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a31f16-e90c-425a-92e6-d6b3e26c0ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:14.774495 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a31f16-e90c-425a-92e6-d6b3e26c0ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:14.793006 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-91a31f16-e90c-425a-92e6-d6b3e26c0ec6 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:14.794263 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-91a31f16-e90c-425a-92e6-d6b3e26c0ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:14.808722 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0ad45-3906-4e77-8f6a-d4f4741a68ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:14.809542 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0ad45-3906-4e77-8f6a-d4f4741a68ae None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:14.809872 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0ad45-3906-4e77-8f6a-d4f4741a68ae None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:14.810159 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0ad45-3906-4e77-8f6a-d4f4741a68ae None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:14.825431 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d0ad45-3906-4e77-8f6a-d4f4741a68ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:52:14.834489 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-14d0ad45-3906-4e77-8f6a-d4f4741a68ae None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 15:52:14.834489 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:14.834489 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:14.834489 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:14.834489 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:14.834489 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:14.834489 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:14.834489 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:14.834489 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:14.834489 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:14.834489 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:14.834489 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:14.834489 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:52:14.834489 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:52:14.834489 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:52:14.834489 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:52:14.835804 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:52:14.835804 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:52:14.835804 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:52:14.835804 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:52:14.835804 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:14.835804 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:14.835804 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:14.835804 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:14.835804 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:14.835804 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:14.835804 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:14.835804 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:14.835804 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:14.835804 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:14.835804 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:14.835804 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:14.837662 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:14.837662 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:14.837662 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:14.837662 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:14.837662 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:52:14.837662 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:52:14.837662 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:52:14.837662 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:52:14.837662 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:52:14.837662 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:52:14.837662 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 15:52:14.837662 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:14.837662 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 477/972] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Jul 21 15:52:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:14.843311 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0ad45-3906-4e77-8f6a-d4f4741a68ae None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:14.883756 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 496/973] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:14 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 15:52:14.912330 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a31f16-e90c-425a-92e6-d6b3e26c0ec6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:14.973765 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0ad45-3906-4e77-8f6a-d4f4741a68ae None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:15.023672 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d882773e-7365-457e-9239-3f6b2e97b5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:15.024252 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d882773e-7365-457e-9239-3f6b2e97b5e3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:15.024541 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d882773e-7365-457e-9239-3f6b2e97b5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:15.024824 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d882773e-7365-457e-9239-3f6b2e97b5e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:15.025618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d882773e-7365-457e-9239-3f6b2e97b5e3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:15.025941 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d882773e-7365-457e-9239-3f6b2e97b5e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:15.027539 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d882773e-7365-457e-9239-3f6b2e97b5e3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:15.031624 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:15.038132 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 478/974] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Jul 21 15:52:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:52:15.039340 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a31f16-e90c-425a-92e6-d6b3e26c0ec6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:15.043679 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d882773e-7365-457e-9239-3f6b2e97b5e3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:15.068034 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:15.079274 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a2afd9-8b1a-4d53-96dd-c28f43b161b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:15.080083 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2afd9-8b1a-4d53-96dd-c28f43b161b6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:15.080368 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2afd9-8b1a-4d53-96dd-c28f43b161b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:15.080661 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2afd9-8b1a-4d53-96dd-c28f43b161b6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:15.081300 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a2afd9-8b1a-4d53-96dd-c28f43b161b6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:15.082727 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a2afd9-8b1a-4d53-96dd-c28f43b161b6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:15.097458 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-47a2afd9-8b1a-4d53-96dd-c28f43b161b6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 15:52:15.097458 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:15.097458 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 15:52:15.097458 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 15:52:15.097458 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 15:52:15.097458 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 15:52:15.097458 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 15:52:15.097458 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:52:15.097458 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 15:52:15.097458 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:15.097458 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 15:52:15.097458 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:15.097458 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:15.097458 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:15.097458 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:15.097458 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:15.097458 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:15.102232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:15.102232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:15.102232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:15.102232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:15.102232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:15.102232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:15.102232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 15:52:15.102232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 15:52:15.102232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 15:52:15.102232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 15:52:15.102232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:15.102232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:15.102232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:15.102232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:15.102232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:15.102232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:15.103131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:15.103131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:15.103131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:15.103131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:15.103131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:15.103131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:15.103131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:15.103131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:15.103131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:15.103131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:15.103131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 15:52:15.103131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:52:15.103131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 15:52:15.103131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:15.103131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 497/975] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jul 21 15:52:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:52:15.105478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a2afd9-8b1a-4d53-96dd-c28f43b161b6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:15.174073 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d882773e-7365-457e-9239-3f6b2e97b5e3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:15.249605 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823dfe56-46c3-43b8-a1ef-50b3784adb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:15.251087 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823dfe56-46c3-43b8-a1ef-50b3784adb66 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:15.251377 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823dfe56-46c3-43b8-a1ef-50b3784adb66 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:15.251750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823dfe56-46c3-43b8-a1ef-50b3784adb66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:15.255665 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a2afd9-8b1a-4d53-96dd-c28f43b161b6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:15.270746 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823dfe56-46c3-43b8-a1ef-50b3784adb66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:15.272251 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823dfe56-46c3-43b8-a1ef-50b3784adb66 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:15.293616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-823dfe56-46c3-43b8-a1ef-50b3784adb66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:52:15.293616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:15.293616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:15.293616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:15.293616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:15.293616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:15.293616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:15.293616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:15.293616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:15.293616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:15.293616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:15.293616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:15.293616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:52:15.293616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:52:15.293616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:52:15.294587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:52:15.294587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:15.294587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:15.294587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:15.294587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:15.294587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:15.294587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:15.294587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:15.294587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:15.294587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:15.294587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:15.294587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:15.294587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:15.294587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:15.294587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:15.294587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:15.294587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:15.295524 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:52:15.295524 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:52:15.295524 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:52:15.295524 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:15.295524 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 479/976] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:52:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:52:15.300506 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823dfe56-46c3-43b8-a1ef-50b3784adb66 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:15.301561 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c51e5-e299-43ad-921e-6f4d9f68ad08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:15.302260 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c51e5-e299-43ad-921e-6f4d9f68ad08 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:15.302542 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c51e5-e299-43ad-921e-6f4d9f68ad08 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:15.302818 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c51e5-e299-43ad-921e-6f4d9f68ad08 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:15.303781 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4c51e5-e299-43ad-921e-6f4d9f68ad08 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:15.304095 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4c51e5-e299-43ad-921e-6f4d9f68ad08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:15.306440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4c51e5-e299-43ad-921e-6f4d9f68ad08 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:15.313838 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 498/977] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jul 21 15:52:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:15.318527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c51e5-e299-43ad-921e-6f4d9f68ad08 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:15.347534 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:15.392030 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:15.431742 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823dfe56-46c3-43b8-a1ef-50b3784adb66 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:15.433331 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c51e5-e299-43ad-921e-6f4d9f68ad08 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:15.473527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9825ad-e15f-4c0d-944a-623fbc7c9865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:15.474562 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9825ad-e15f-4c0d-944a-623fbc7c9865 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:15.475001 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9825ad-e15f-4c0d-944a-623fbc7c9865 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:15.475284 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9825ad-e15f-4c0d-944a-623fbc7c9865 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:15.476002 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9825ad-e15f-4c0d-944a-623fbc7c9865 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:15.476316 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9825ad-e15f-4c0d-944a-623fbc7c9865 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:15.477741 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9825ad-e15f-4c0d-944a-623fbc7c9865 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:15.483904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f4e3f-37f2-4342-abf1-f1ddfb97e2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:15.491744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f4e3f-37f2-4342-abf1-f1ddfb97e2fd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:15.491744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f4e3f-37f2-4342-abf1-f1ddfb97e2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:15.491744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f4e3f-37f2-4342-abf1-f1ddfb97e2fd None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:15.491744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0f4e3f-37f2-4342-abf1-f1ddfb97e2fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:15.491744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0f4e3f-37f2-4342-abf1-f1ddfb97e2fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:15.492208 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 499/978] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jul 21 15:52:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:52:15.492291 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0f4e3f-37f2-4342-abf1-f1ddfb97e2fd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:15.492785 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9825ad-e15f-4c0d-944a-623fbc7c9865 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:15.504826 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 480/979] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:52:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:52:15.512760 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f4e3f-37f2-4342-abf1-f1ddfb97e2fd None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:15.608677 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:15.626587 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9825ad-e15f-4c0d-944a-623fbc7c9865 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:15.637673 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f4e3f-37f2-4342-abf1-f1ddfb97e2fd None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:15.666077 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb2515-e509-4987-850d-3fc9e4abae84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:15.666764 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb2515-e509-4987-850d-3fc9e4abae84 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:15.667044 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb2515-e509-4987-850d-3fc9e4abae84 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:15.667318 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb2515-e509-4987-850d-3fc9e4abae84 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:15.668234 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb2515-e509-4987-850d-3fc9e4abae84 None admin] RBAC: Adding query filter params (interface=admin, service_id=fcc4a96e80af41b395ff893481f7e957, region_id=RegionOne) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:15.668552 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb2515-e509-4987-850d-3fc9e4abae84 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:15.669985 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb2515-e509-4987-850d-3fc9e4abae84 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:15.673624 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:15.681414 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 500/980] 10.222.0.52 () {56 vars in 1249 bytes} [Thu Jul 21 15:52:15 2022] GET /identity/v3/endpoints?service_id=fcc4a96e80af41b395ff893481f7e957&interface=admin®ion_id=RegionOne => generated 193 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:52:15.682590 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb2515-e509-4987-850d-3fc9e4abae84 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:15.685143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2dec5-aab7-45e6-9593-c243a2553ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:15.686701 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2dec5-aab7-45e6-9593-c243a2553ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:15.687064 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2dec5-aab7-45e6-9593-c243a2553ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:15.687490 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2dec5-aab7-45e6-9593-c243a2553ef9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:15.696576 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f2dec5-aab7-45e6-9593-c243a2553ef9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:15.698086 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f2dec5-aab7-45e6-9593-c243a2553ef9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:15.719203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-e8f2dec5-aab7-45e6-9593-c243a2553ef9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 15:52:15.719203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:15.719203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:15.719203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:15.719203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:15.719203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:15.719203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:15.719203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:15.719203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:15.719203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:15.719203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:15.719203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:15.719203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:52:15.719203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:52:15.719203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:52:15.719203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:52:15.720416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:15.720416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:15.720416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:52:15.720416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:52:15.720416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:52:15.720416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:52:15.720416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:15.720416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:15.720416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:15.720416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:15.720416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:15.720416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:15.720416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:15.720416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:15.720416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:15.720416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:15.720416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:15.720416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:15.722519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:15.722519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:15.722519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:52:15.722519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:52:15.722519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:52:15.722519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:52:15.722519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:52:15.722519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:52:15.722519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 15:52:15.722519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:15.722519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 481/981] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jul 21 15:52:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:52:15.722519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2dec5-aab7-45e6-9593-c243a2553ef9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:15.807139 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb2515-e509-4987-850d-3fc9e4abae84 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:15.846336 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296cb0f-2503-445e-81ee-f9e92a0e7054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:15.847252 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296cb0f-2503-445e-81ee-f9e92a0e7054 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:15.847547 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296cb0f-2503-445e-81ee-f9e92a0e7054 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:15.847809 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296cb0f-2503-445e-81ee-f9e92a0e7054 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:15.848478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2296cb0f-2503-445e-81ee-f9e92a0e7054 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:15.849898 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2296cb0f-2503-445e-81ee-f9e92a0e7054 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:15.851883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2dec5-aab7-45e6-9593-c243a2553ef9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:15.865704 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 501/982] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jul 21 15:52:15 2022] GET /identity/v3/services => generated 1727 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 15:52:15.905648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b62ff-4e72-4436-b29d-f69e96aa1241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:15.906355 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b62ff-4e72-4436-b29d-f69e96aa1241 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:15.906644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b62ff-4e72-4436-b29d-f69e96aa1241 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:15.906932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b62ff-4e72-4436-b29d-f69e96aa1241 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:15.907810 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b62ff-4e72-4436-b29d-f69e96aa1241 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:15.908163 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b62ff-4e72-4436-b29d-f69e96aa1241 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:15.909879 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b62ff-4e72-4436-b29d-f69e96aa1241 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:15.931122 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 482/983] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Jul 21 15:52:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:52:15.939408 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296cb0f-2503-445e-81ee-f9e92a0e7054 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:15.958371 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:16.069233 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296cb0f-2503-445e-81ee-f9e92a0e7054 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:16.077997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3f206-386b-4aa5-b214-1be207084837 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:16.078473 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3f206-386b-4aa5-b214-1be207084837 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:16.078925 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3f206-386b-4aa5-b214-1be207084837 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:16.081364 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 483/984] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 15:52:16.086724 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860fac9-7342-4331-97e6-4d9271a72bf5 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:16.087090 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860fac9-7342-4331-97e6-4d9271a72bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:16.087285 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860fac9-7342-4331-97e6-4d9271a72bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:16.109459 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1860fac9-7342-4331-97e6-4d9271a72bf5 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:16.110700 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1860fac9-7342-4331-97e6-4d9271a72bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:16.115960 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb0c31-a6a7-40ab-8518-8e8012778102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:16.116559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb0c31-a6a7-40ab-8518-8e8012778102 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:16.116846 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb0c31-a6a7-40ab-8518-8e8012778102 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:16.117128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb0c31-a6a7-40ab-8518-8e8012778102 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:16.177055 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:16.207027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdb0c31-a6a7-40ab-8518-8e8012778102 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:16.207522 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 484/985] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:16 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:52:16.211276 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdb0c31-a6a7-40ab-8518-8e8012778102 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:16.229444 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:16.233284 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1860fac9-7342-4331-97e6-4d9271a72bf5 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:16.332281 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 502/986] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:52:15 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 15:52:16.338311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb0c31-a6a7-40ab-8518-8e8012778102 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:16.375574 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1860fac9-7342-4331-97e6-4d9271a72bf5 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:16.420129 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9400fd5a-f140-42b7-a9ff-681ca8ae3e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:16.420938 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400fd5a-f140-42b7-a9ff-681ca8ae3e7c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:16.421224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400fd5a-f140-42b7-a9ff-681ca8ae3e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:16.421513 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400fd5a-f140-42b7-a9ff-681ca8ae3e7c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:16.422204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9400fd5a-f140-42b7-a9ff-681ca8ae3e7c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:16.423590 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9400fd5a-f140-42b7-a9ff-681ca8ae3e7c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:16.436932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-9400fd5a-f140-42b7-a9ff-681ca8ae3e7c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 15:52:16.436932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:16.436932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 15:52:16.436932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 15:52:16.436932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 15:52:16.436932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 15:52:16.436932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 15:52:16.436932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:52:16.436932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 15:52:16.436932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:16.436932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 15:52:16.436932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:16.436932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:16.436932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:16.436932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:16.436932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:16.436932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:16.442330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:16.442330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:16.442330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:16.442330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:16.442330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:16.442330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:16.442330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 15:52:16.442330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 15:52:16.442330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 15:52:16.442330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 15:52:16.442330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:16.442330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:16.442330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:16.442330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:16.442330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:16.442330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:16.443199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:16.443199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:16.443199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:16.443199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:16.443199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:16.443199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:16.443199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:16.443199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:16.443199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:16.443199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:16.443199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 15:52:16.443199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:52:16.443199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 15:52:16.443199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:16.443199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 485/987] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:52:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:16.446991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9400fd5a-f140-42b7-a9ff-681ca8ae3e7c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:16.471291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb0c31-a6a7-40ab-8518-8e8012778102 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:16.487816 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:16.496739 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:16.518579 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127b903-a0f9-4d6e-bddf-b9a9b00e53fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:16.519392 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127b903-a0f9-4d6e-bddf-b9a9b00e53fb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:16.519681 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127b903-a0f9-4d6e-bddf-b9a9b00e53fb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:16.519961 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127b903-a0f9-4d6e-bddf-b9a9b00e53fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:16.532263 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127b903-a0f9-4d6e-bddf-b9a9b00e53fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:16.533771 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127b903-a0f9-4d6e-bddf-b9a9b00e53fb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:16.554887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-2127b903-a0f9-4d6e-bddf-b9a9b00e53fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:52:16.554887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:16.554887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:16.554887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:16.554887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:16.554887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:16.554887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:16.554887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:16.554887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:16.554887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:16.554887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:16.554887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:16.554887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:52:16.554887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:52:16.554887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:52:16.556056 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:52:16.556056 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:16.556056 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:16.556056 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:16.556056 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:16.556056 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:16.556056 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:16.556056 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:16.556056 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:16.556056 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:16.556056 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:16.556056 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:16.556056 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:16.556056 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:16.556056 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:16.556056 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:16.556056 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:16.557100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:52:16.557100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:52:16.557100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:52:16.557100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:16.557100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 503/988] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:52:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:52:16.563008 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127b903-a0f9-4d6e-bddf-b9a9b00e53fb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:16.566743 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9400fd5a-f140-42b7-a9ff-681ca8ae3e7c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:16.608762 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207a871-e7a6-4cb4-b83c-126bbbed80a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:16.609451 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207a871-e7a6-4cb4-b83c-126bbbed80a9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:16.609783 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207a871-e7a6-4cb4-b83c-126bbbed80a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:16.610060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207a871-e7a6-4cb4-b83c-126bbbed80a9 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:16.610799 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207a871-e7a6-4cb4-b83c-126bbbed80a9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:16.611110 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207a871-e7a6-4cb4-b83c-126bbbed80a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:16.612519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207a871-e7a6-4cb4-b83c-126bbbed80a9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:16.618365 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 486/989] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 21 15:52:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:16.627046 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207a871-e7a6-4cb4-b83c-126bbbed80a9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:16.688549 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127b903-a0f9-4d6e-bddf-b9a9b00e53fb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:16.730895 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325bf7d4-3133-4cb2-8b50-14215d875090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:16.731576 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325bf7d4-3133-4cb2-8b50-14215d875090 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:16.731862 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325bf7d4-3133-4cb2-8b50-14215d875090 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:16.732182 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325bf7d4-3133-4cb2-8b50-14215d875090 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:16.733053 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325bf7d4-3133-4cb2-8b50-14215d875090 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:16.733371 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325bf7d4-3133-4cb2-8b50-14215d875090 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:16.734928 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325bf7d4-3133-4cb2-8b50-14215d875090 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:16.737313 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207a871-e7a6-4cb4-b83c-126bbbed80a9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:16.744517 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:16.750263 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 504/990] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:52:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:52:16.757839 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325bf7d4-3133-4cb2-8b50-14215d875090 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:16.778780 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c0d5f8-6236-4323-8246-fb1848fd10f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:16.780028 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c0d5f8-6236-4323-8246-fb1848fd10f7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:16.780317 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c0d5f8-6236-4323-8246-fb1848fd10f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:16.780601 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c0d5f8-6236-4323-8246-fb1848fd10f7 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:16.781323 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c0d5f8-6236-4323-8246-fb1848fd10f7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:16.781664 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c0d5f8-6236-4323-8246-fb1848fd10f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:16.783070 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c0d5f8-6236-4323-8246-fb1848fd10f7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:16.788558 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 487/991] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 21 15:52:16 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:52:16.794440 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c0d5f8-6236-4323-8246-fb1848fd10f7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:16.798856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:16.900769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325bf7d4-3133-4cb2-8b50-14215d875090 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:16.922818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c0d5f8-6236-4323-8246-fb1848fd10f7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:16.938193 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae4c83d-2b51-4b35-9b4e-7dc9dc28d5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:16.939136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4c83d-2b51-4b35-9b4e-7dc9dc28d5c2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:16.939419 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4c83d-2b51-4b35-9b4e-7dc9dc28d5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:16.939695 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4c83d-2b51-4b35-9b4e-7dc9dc28d5c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:16.947087 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae4c83d-2b51-4b35-9b4e-7dc9dc28d5c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:16.948550 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae4c83d-2b51-4b35-9b4e-7dc9dc28d5c2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:16.964249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-9ae4c83d-2b51-4b35-9b4e-7dc9dc28d5c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:52:16.964249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:16.964249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:16.964249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:16.964249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:16.964249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:16.964249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:16.964249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:16.964249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:16.964249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:16.964249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:16.964249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:16.964249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:52:16.964249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:52:16.964249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:52:16.965270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:52:16.965270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:16.965270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:16.965270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:16.965270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:16.965270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:16.965270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:16.965270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:16.965270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:16.965270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:16.965270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:16.965270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:16.965270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:16.965270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:16.965270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:16.965270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:16.965270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:16.966381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:52:16.966381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:52:16.966381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:52:16.966381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:52:16.966381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:52:16.966381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:52:16.966381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:52:16.966381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:16.970224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e0c70-7884-4c1d-bbdd-04ca79670d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:16.970979 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e0c70-7884-4c1d-bbdd-04ca79670d7e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:16.971334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 505/992] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:52:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:52:16.971445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e0c70-7884-4c1d-bbdd-04ca79670d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:16.971615 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e0c70-7884-4c1d-bbdd-04ca79670d7e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:16.972630 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae4c83d-2b51-4b35-9b4e-7dc9dc28d5c2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:16.973137 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873e0c70-7884-4c1d-bbdd-04ca79670d7e None admin] RBAC: Adding query filter params (interface=public, service_id=ce018447482a487e9ab170bfd2c11576, region_id=RegionOne) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:16.973702 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873e0c70-7884-4c1d-bbdd-04ca79670d7e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:16.981676 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873e0c70-7884-4c1d-bbdd-04ca79670d7e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:16.987889 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 488/993] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Jul 21 15:52:16 2022] GET /identity/v3/endpoints?service_id=ce018447482a487e9ab170bfd2c11576&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:16.993727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e0c70-7884-4c1d-bbdd-04ca79670d7e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:17.038588 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:17.089037 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:17.112697 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae4c83d-2b51-4b35-9b4e-7dc9dc28d5c2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:17.118052 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e0c70-7884-4c1d-bbdd-04ca79670d7e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:17.156499 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16913ae5-19b7-48b8-9dcc-1fd1bba20dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:17.157376 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16913ae5-19b7-48b8-9dcc-1fd1bba20dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:17.157709 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16913ae5-19b7-48b8-9dcc-1fd1bba20dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:17.157996 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16913ae5-19b7-48b8-9dcc-1fd1bba20dc2 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:17.158733 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16913ae5-19b7-48b8-9dcc-1fd1bba20dc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:17.160169 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16913ae5-19b7-48b8-9dcc-1fd1bba20dc2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:17.177609 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 489/994] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jul 21 15:52:16 2022] GET /identity/v3/services => generated 1727 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 15:52:17.177827 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91942fff-5e5e-4433-b3d7-34430b523576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:17.177827 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91942fff-5e5e-4433-b3d7-34430b523576 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:17.177827 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91942fff-5e5e-4433-b3d7-34430b523576 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:17.177827 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91942fff-5e5e-4433-b3d7-34430b523576 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:17.178194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91942fff-5e5e-4433-b3d7-34430b523576 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:17.178194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91942fff-5e5e-4433-b3d7-34430b523576 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:17.178194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91942fff-5e5e-4433-b3d7-34430b523576 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:17.191542 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 506/995] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:52:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:52:17.197437 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16913ae5-19b7-48b8-9dcc-1fd1bba20dc2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:17.342624 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:17.348206 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16913ae5-19b7-48b8-9dcc-1fd1bba20dc2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:17.388835 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e6f96-178d-4ce6-9cad-8eefb61d131c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:17.389782 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e6f96-178d-4ce6-9cad-8eefb61d131c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:17.389943 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e6f96-178d-4ce6-9cad-8eefb61d131c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:17.390143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e6f96-178d-4ce6-9cad-8eefb61d131c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:17.390964 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e6f96-178d-4ce6-9cad-8eefb61d131c None admin] RBAC: Adding query filter params (role.id=abd0095f0bf64a0c8e694dbf50d6fd9a, scope.project.id=bb1ac00308714c39a79aae09fbb73021, user.id=6edbad98ed0f4823b9955f9f0e270149) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:17.391285 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e6f96-178d-4ce6-9cad-8eefb61d131c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:17.392747 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e6f96-178d-4ce6-9cad-8eefb61d131c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:17.405678 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 490/996] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:52:17 2022] GET /identity/v3/role_assignments?user.id=6edbad98ed0f4823b9955f9f0e270149&scope.project.id=bb1ac00308714c39a79aae09fbb73021&role.id=abd0095f0bf64a0c8e694dbf50d6fd9a => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:17.951840 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2f590-4993-4a34-bfe6-016c5000ea9b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:17.952594 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2f590-4993-4a34-bfe6-016c5000ea9b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:17.952594 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2f590-4993-4a34-bfe6-016c5000ea9b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:17.954427 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 507/997] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:52:17.959855 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbfa1b-f5bb-4ec0-a7bd-d6d4d6717440 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:17.960403 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbfa1b-f5bb-4ec0-a7bd-d6d4d6717440 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:17.960403 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbfa1b-f5bb-4ec0-a7bd-d6d4d6717440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:17.977620 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3dfbfa1b-f5bb-4ec0-a7bd-d6d4d6717440 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:17.978807 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3dfbfa1b-f5bb-4ec0-a7bd-d6d4d6717440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:18.064411 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 491/998] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:17 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:52:18.087997 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e2f590-4993-4a34-bfe6-016c5000ea9b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:18.203650 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e2f590-4993-4a34-bfe6-016c5000ea9b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:18.250253 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e4cb5-23d2-4419-86ac-fe646c3077a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:18.251061 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e4cb5-23d2-4419-86ac-fe646c3077a9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:18.251345 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e4cb5-23d2-4419-86ac-fe646c3077a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:18.251624 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e4cb5-23d2-4419-86ac-fe646c3077a9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:18.252317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e4cb5-23d2-4419-86ac-fe646c3077a9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:18.253848 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e4cb5-23d2-4419-86ac-fe646c3077a9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:18.278286 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-904e4cb5-23d2-4419-86ac-fe646c3077a9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 15:52:18.278286 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:18.278286 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 15:52:18.278286 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 15:52:18.278286 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 15:52:18.278286 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 15:52:18.278286 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 15:52:18.278286 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:52:18.278286 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 15:52:18.278286 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:18.278286 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 15:52:18.278286 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:18.278286 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:18.278286 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:18.278286 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:18.278286 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:18.278286 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:18.280806 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:18.280806 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:18.280806 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:18.280806 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:18.280806 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:18.280806 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:18.280806 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 15:52:18.280806 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 15:52:18.280806 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 15:52:18.280806 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 15:52:18.280806 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:18.280806 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:18.280806 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:18.280806 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:18.280806 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:18.280806 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:18.284637 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:18.284637 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:18.284637 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:18.284637 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:18.284637 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:18.284637 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:18.284637 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:18.284637 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:18.284637 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:18.284637 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:18.284637 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 15:52:18.284637 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:52:18.284637 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 15:52:18.284637 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:18.284637 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 508/999] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jul 21 15:52:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:18.289067 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfbfa1b-f5bb-4ec0-a7bd-d6d4d6717440 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:18.402677 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:18.414706 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfbfa1b-f5bb-4ec0-a7bd-d6d4d6717440 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:18.450466 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95401710-16e2-47dd-973a-a76eb9fa3fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:18.451663 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95401710-16e2-47dd-973a-a76eb9fa3fea None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:18.451942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95401710-16e2-47dd-973a-a76eb9fa3fea None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:18.452205 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95401710-16e2-47dd-973a-a76eb9fa3fea None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:18.452925 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95401710-16e2-47dd-973a-a76eb9fa3fea None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:18.453231 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95401710-16e2-47dd-973a-a76eb9fa3fea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:18.454661 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95401710-16e2-47dd-973a-a76eb9fa3fea None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:18.464677 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 492/1000] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jul 21 15:52:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:18.465856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e4cb5-23d2-4419-86ac-fe646c3077a9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:18.480181 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:18.582381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e4cb5-23d2-4419-86ac-fe646c3077a9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:18.619807 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ebd1f-1222-4c46-9283-7a995bf03583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:18.620093 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ebd1f-1222-4c46-9283-7a995bf03583 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:18.620416 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ebd1f-1222-4c46-9283-7a995bf03583 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:18.620602 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ebd1f-1222-4c46-9283-7a995bf03583 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:18.621269 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ebd1f-1222-4c46-9283-7a995bf03583 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:18.621602 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ebd1f-1222-4c46-9283-7a995bf03583 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:18.623517 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ebd1f-1222-4c46-9283-7a995bf03583 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:18.633673 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 509/1001] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jul 21 15:52:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:18.634632 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95401710-16e2-47dd-973a-a76eb9fa3fea None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:18.749192 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95401710-16e2-47dd-973a-a76eb9fa3fea None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:18.799947 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47cf16-349a-42e9-9f33-6a95ca6204e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:18.800994 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47cf16-349a-42e9-9f33-6a95ca6204e6 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:18.801482 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47cf16-349a-42e9-9f33-6a95ca6204e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:18.801956 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47cf16-349a-42e9-9f33-6a95ca6204e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:18.803139 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc47cf16-349a-42e9-9f33-6a95ca6204e6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:18.805401 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc47cf16-349a-42e9-9f33-6a95ca6204e6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:18.828707 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 493/1002] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jul 21 15:52:18 2022] POST /identity/v3/endpoints => generated 325 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 15:52:19.288488 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf35d7-b96a-4f90-83c8-080d89c16871 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:19.289008 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf35d7-b96a-4f90-83c8-080d89c16871 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:19.289119 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf35d7-b96a-4f90-83c8-080d89c16871 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:19.290583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 510/1003] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:52:19.295891 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737a778-6e4e-432a-83a5-38c903b8a6b3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:19.296355 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737a778-6e4e-432a-83a5-38c903b8a6b3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:19.296465 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737a778-6e4e-432a-83a5-38c903b8a6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:19.322705 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9737a778-6e4e-432a-83a5-38c903b8a6b3 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:19.323924 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9737a778-6e4e-432a-83a5-38c903b8a6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:19.337269 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:19.413444 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 494/1004] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:19 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:52:19.436646 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf35d7-b96a-4f90-83c8-080d89c16871 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:19.527302 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:19.556527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf35d7-b96a-4f90-83c8-080d89c16871 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:19.556875 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22db8bb-4fee-43fe-bfaa-bca452e13649 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:19.557056 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22db8bb-4fee-43fe-bfaa-bca452e13649 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:19.557331 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22db8bb-4fee-43fe-bfaa-bca452e13649 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:19.558901 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 495/1005] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 15:52:19.564416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79202850-7be1-4bb6-930a-e6c82a3c0d5d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:19.564718 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79202850-7be1-4bb6-930a-e6c82a3c0d5d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:19.565008 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79202850-7be1-4bb6-930a-e6c82a3c0d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:19.583057 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-79202850-7be1-4bb6-930a-e6c82a3c0d5d None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:19.584425 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-79202850-7be1-4bb6-930a-e6c82a3c0d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:19.596665 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27847952-87c6-43c9-80d3-fea1cd64dd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:19.597763 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27847952-87c6-43c9-80d3-fea1cd64dd5b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:19.598050 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27847952-87c6-43c9-80d3-fea1cd64dd5b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:19.598328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27847952-87c6-43c9-80d3-fea1cd64dd5b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:19.599006 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27847952-87c6-43c9-80d3-fea1cd64dd5b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:19.600425 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27847952-87c6-43c9-80d3-fea1cd64dd5b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:19.613626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-27847952-87c6-43c9-80d3-fea1cd64dd5b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 15:52:19.613626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:19.613626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 15:52:19.613626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 15:52:19.613626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 15:52:19.613626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 15:52:19.613626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 15:52:19.613626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:52:19.613626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 15:52:19.613626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:19.613626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 15:52:19.613626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:19.613626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:19.613626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:19.613626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:19.613626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:19.613626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:19.615676 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:19.615676 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:19.615676 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:19.615676 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:19.615676 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:19.615676 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:19.615676 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 15:52:19.615676 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 15:52:19.615676 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 15:52:19.615676 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 15:52:19.615676 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:19.615676 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:19.615676 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:19.615676 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:19.615676 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:19.615676 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:19.616845 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:19.616845 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:19.616845 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:19.616845 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:19.616845 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:19.616845 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:19.616845 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:19.616845 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:19.616845 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:19.616845 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:19.616845 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 15:52:19.616845 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:52:19.616845 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 15:52:19.616845 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:19.619942 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 511/1006] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:52:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:19.620862 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27847952-87c6-43c9-80d3-fea1cd64dd5b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:19.694565 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 496/1007] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:19 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:52:19.721460 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79202850-7be1-4bb6-930a-e6c82a3c0d5d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:19.748324 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27847952-87c6-43c9-80d3-fea1cd64dd5b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:19.787022 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a411f-da4b-409e-966d-632b4be29b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:19.787607 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a411f-da4b-409e-966d-632b4be29b73 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:19.787890 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a411f-da4b-409e-966d-632b4be29b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:19.788168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a411f-da4b-409e-966d-632b4be29b73 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:19.788881 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a411f-da4b-409e-966d-632b4be29b73 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:19.789191 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a411f-da4b-409e-966d-632b4be29b73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:19.790616 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a411f-da4b-409e-966d-632b4be29b73 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:19.803043 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 512/1008] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 21 15:52:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:19.804013 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a411f-da4b-409e-966d-632b4be29b73 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:19.806916 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:19.828881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:19.869740 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79202850-7be1-4bb6-930a-e6c82a3c0d5d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:19.909802 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57434be6-e259-4c06-9f06-d5697057d7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:19.910611 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57434be6-e259-4c06-9f06-d5697057d7cb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:19.910894 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57434be6-e259-4c06-9f06-d5697057d7cb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:19.911168 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57434be6-e259-4c06-9f06-d5697057d7cb None admin] PATH_INFO: `/v3/services/image` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:19.911835 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57434be6-e259-4c06-9f06-d5697057d7cb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:19.913341 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57434be6-e259-4c06-9f06-d5697057d7cb None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:19.921515 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a411f-da4b-409e-966d-632b4be29b73 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:19.925868 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-57434be6-e259-4c06-9f06-d5697057d7cb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 15:52:19.925868 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:19.925868 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 15:52:19.925868 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 15:52:19.925868 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 15:52:19.925868 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 15:52:19.925868 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 15:52:19.925868 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:52:19.925868 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 15:52:19.925868 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:19.925868 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 15:52:19.925868 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:19.925868 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:19.925868 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:19.925868 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:19.925868 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:19.925868 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:19.927664 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:19.927664 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:19.927664 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:19.927664 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:19.927664 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:19.927664 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:19.927664 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 15:52:19.927664 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 15:52:19.927664 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 15:52:19.927664 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 15:52:19.927664 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:19.927664 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:19.927664 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:19.927664 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:19.927664 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:19.927664 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:19.929182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:19.929182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:19.929182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:19.929182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:19.929182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:19.929182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:19.929182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:19.929182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:19.929182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:19.929182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:19.929182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 15:52:19.929182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:52:19.929182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 15:52:19.929182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:19.932046 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 497/1009] 10.222.0.52 () {56 vars in 1105 bytes} [Thu Jul 21 15:52:19 2022] GET /identity/v3/services/image => generated 86 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:19.933423 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57434be6-e259-4c06-9f06-d5697057d7cb None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:19.976050 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5e5c8-1770-45ac-ab83-fc7ae024e35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:19.976935 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5e5c8-1770-45ac-ab83-fc7ae024e35e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:19.977349 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5e5c8-1770-45ac-ab83-fc7ae024e35e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:19.977806 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5e5c8-1770-45ac-ab83-fc7ae024e35e None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:19.978701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e5e5c8-1770-45ac-ab83-fc7ae024e35e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:19.979709 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e5e5c8-1770-45ac-ab83-fc7ae024e35e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:19.988906 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e5e5c8-1770-45ac-ab83-fc7ae024e35e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:20.000952 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 513/1010] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 21 15:52:19 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:52:20.003309 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5e5c8-1770-45ac-ab83-fc7ae024e35e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:20.083733 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57434be6-e259-4c06-9f06-d5697057d7cb None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:20.102520 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:20.126913 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d45cde-b082-42e7-98f0-cde696312c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:20.127480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d45cde-b082-42e7-98f0-cde696312c14 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:20.127767 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d45cde-b082-42e7-98f0-cde696312c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:20.128044 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d45cde-b082-42e7-98f0-cde696312c14 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:20.128762 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d45cde-b082-42e7-98f0-cde696312c14 None admin] RBAC: Adding query filter params (name=image) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:20.129074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d45cde-b082-42e7-98f0-cde696312c14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:20.130503 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d45cde-b082-42e7-98f0-cde696312c14 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:20.132657 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:20.142324 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 498/1011] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:52:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:20.143510 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d45cde-b082-42e7-98f0-cde696312c14 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:20.147626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5e5c8-1770-45ac-ab83-fc7ae024e35e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:20.194685 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571f2ee-9c46-49ea-b0dc-42ea0e5d1a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:20.195357 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571f2ee-9c46-49ea-b0dc-42ea0e5d1a7f None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:20.195629 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571f2ee-9c46-49ea-b0dc-42ea0e5d1a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:20.195904 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571f2ee-9c46-49ea-b0dc-42ea0e5d1a7f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:20.196672 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0571f2ee-9c46-49ea-b0dc-42ea0e5d1a7f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:20.198108 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0571f2ee-9c46-49ea-b0dc-42ea0e5d1a7f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:20.229609 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 514/1012] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jul 21 15:52:20 2022] POST /identity/v3/endpoints => generated 334 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 15:52:20.277481 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d45cde-b082-42e7-98f0-cde696312c14 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:20.347778 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c512626-bddc-44fb-acb3-7612c9ac49d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:20.348821 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c512626-bddc-44fb-acb3-7612c9ac49d7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:20.349302 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c512626-bddc-44fb-acb3-7612c9ac49d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:20.349725 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c512626-bddc-44fb-acb3-7612c9ac49d7 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:20.350490 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c512626-bddc-44fb-acb3-7612c9ac49d7 None admin] RBAC: Adding query filter params (type=image) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:20.350810 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c512626-bddc-44fb-acb3-7612c9ac49d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:20.352243 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c512626-bddc-44fb-acb3-7612c9ac49d7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:20.366785 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 499/1013] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:52:20 2022] GET /identity/v3/services?type=image => generated 122 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:52:20.949188 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fb779-a7fa-4973-a645-08ee1d4f6b38 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:20.949744 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fb779-a7fa-4973-a645-08ee1d4f6b38 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:20.949744 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fb779-a7fa-4973-a645-08ee1d4f6b38 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:20.951824 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 515/1014] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:52:20.957114 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e5a82-6a02-4493-bcd6-846e90648c3e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:20.957401 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e5a82-6a02-4493-bcd6-846e90648c3e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:20.957712 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e5a82-6a02-4493-bcd6-846e90648c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:20.974936 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-945e5a82-6a02-4493-bcd6-846e90648c3e None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:20.976172 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-945e5a82-6a02-4493-bcd6-846e90648c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:21.046810 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:21.059412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 500/1015] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:20 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:52:21.083275 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fb779-a7fa-4973-a645-08ee1d4f6b38 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:21.202715 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fb779-a7fa-4973-a645-08ee1d4f6b38 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:21.227217 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:21.251607 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedc39ed-087f-46ce-a884-b263b83eeec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:21.252423 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc39ed-087f-46ce-a884-b263b83eeec7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:21.252710 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc39ed-087f-46ce-a884-b263b83eeec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:21.252987 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc39ed-087f-46ce-a884-b263b83eeec7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:21.254235 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedc39ed-087f-46ce-a884-b263b83eeec7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:21.255767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedc39ed-087f-46ce-a884-b263b83eeec7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:21.257682 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 516/1016] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:52:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:21.264113 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e5a82-6a02-4493-bcd6-846e90648c3e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:21.381817 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e5a82-6a02-4493-bcd6-846e90648c3e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:21.421853 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001bf31c-3273-4c13-855e-f97ad2e90829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:21.422525 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001bf31c-3273-4c13-855e-f97ad2e90829 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:21.422814 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001bf31c-3273-4c13-855e-f97ad2e90829 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:21.423094 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001bf31c-3273-4c13-855e-f97ad2e90829 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:21.423822 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001bf31c-3273-4c13-855e-f97ad2e90829 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:21.425274 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001bf31c-3273-4c13-855e-f97ad2e90829 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:21.454773 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 501/1017] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 21 15:52:21 2022] POST /identity/v3/projects => generated 311 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:52:22.427931 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955358a-462b-4587-9cf3-ea09903bacd8 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:22.428649 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955358a-462b-4587-9cf3-ea09903bacd8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:22.428649 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955358a-462b-4587-9cf3-ea09903bacd8 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:22.430745 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 517/1018] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 15:52:22.438528 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00083b-5e2e-4286-8c85-a1068e6d468d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:22.439300 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00083b-5e2e-4286-8c85-a1068e6d468d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:22.439459 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00083b-5e2e-4286-8c85-a1068e6d468d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:22.458430 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3f00083b-5e2e-4286-8c85-a1068e6d468d None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:22.459646 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3f00083b-5e2e-4286-8c85-a1068e6d468d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:22.482063 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9578c-8d30-4c81-b044-f0bc122847a9 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:22.482465 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9578c-8d30-4c81-b044-f0bc122847a9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:22.482670 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9578c-8d30-4c81-b044-f0bc122847a9 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:22.483993 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 518/1019] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:52:22.489468 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a170885-aada-4e70-993e-262eec28c675 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:22.489791 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a170885-aada-4e70-993e-262eec28c675 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:22.489982 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a170885-aada-4e70-993e-262eec28c675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:22.510128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4a170885-aada-4e70-993e-262eec28c675 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:22.510128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4a170885-aada-4e70-993e-262eec28c675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:22.514040 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:22.565052 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 502/1020] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:22 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:52:22.588896 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f00083b-5e2e-4286-8c85-a1068e6d468d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:22.591337 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 519/1021] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:22 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 15:52:22.616301 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a170885-aada-4e70-993e-262eec28c675 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:22.707552 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f00083b-5e2e-4286-8c85-a1068e6d468d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:22.746858 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eeec82-d6da-47fc-b0d2-58d96bcb9212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:22.747632 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eeec82-d6da-47fc-b0d2-58d96bcb9212 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:22.747915 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eeec82-d6da-47fc-b0d2-58d96bcb9212 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:22.748191 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eeec82-d6da-47fc-b0d2-58d96bcb9212 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:22.752956 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a170885-aada-4e70-993e-262eec28c675 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:22.760304 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eeec82-d6da-47fc-b0d2-58d96bcb9212 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:52:22.770209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-54eeec82-d6da-47fc-b0d2-58d96bcb9212 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 15:52:22.770209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:22.770209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:22.770209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:22.770209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:22.770209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:22.770209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:22.770209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:22.770209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:22.770209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:22.770209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:22.770209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:22.770209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:52:22.770209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:52:22.770209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:52:22.770209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:52:22.771221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:52:22.771221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:52:22.771221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:52:22.771221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:52:22.771221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:22.771221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:22.771221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:22.771221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:22.771221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:22.771221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:22.771221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:22.771221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:22.771221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:22.771221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:22.771221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:22.771221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:22.772228 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:22.772228 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:22.772228 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:22.772228 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:22.772228 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:52:22.772228 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:52:22.772228 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:52:22.772228 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:52:22.772228 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:52:22.772228 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:52:22.772228 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 15:52:22.772228 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:22.772228 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 503/1022] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Jul 21 15:52:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:52:22.777603 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eeec82-d6da-47fc-b0d2-58d96bcb9212 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:22.778103 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:22.804463 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3dfb4-4b6a-4a38-b357-8f7fd99a1e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:22.805674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3dfb4-4b6a-4a38-b357-8f7fd99a1e11 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:22.806129 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3dfb4-4b6a-4a38-b357-8f7fd99a1e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:22.807149 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3dfb4-4b6a-4a38-b357-8f7fd99a1e11 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:22.807899 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe3dfb4-4b6a-4a38-b357-8f7fd99a1e11 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:22.809300 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe3dfb4-4b6a-4a38-b357-8f7fd99a1e11 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:22.813425 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:22.825336 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 520/1023] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 21 15:52:22 2022] POST /identity/v3/services => generated 242 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 15:52:22.908562 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eeec82-d6da-47fc-b0d2-58d96bcb9212 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:22.963790 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998ab01-dd7d-4328-99a7-84e451f19977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:22.964789 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998ab01-dd7d-4328-99a7-84e451f19977 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:22.965283 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998ab01-dd7d-4328-99a7-84e451f19977 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:22.965769 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998ab01-dd7d-4328-99a7-84e451f19977 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:22.967339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998ab01-dd7d-4328-99a7-84e451f19977 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:22.967834 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998ab01-dd7d-4328-99a7-84e451f19977 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:22.969602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998ab01-dd7d-4328-99a7-84e451f19977 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:22.984008 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 504/1024] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Jul 21 15:52:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:52:22.990810 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3dfb4-4b6a-4a38-b357-8f7fd99a1e11 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:23.112796 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3dfb4-4b6a-4a38-b357-8f7fd99a1e11 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:23.155592 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8b5be-1fb6-4cb6-ae11-12996d4d641b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:23.156394 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8b5be-1fb6-4cb6-ae11-12996d4d641b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:23.156680 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8b5be-1fb6-4cb6-ae11-12996d4d641b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:23.156957 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8b5be-1fb6-4cb6-ae11-12996d4d641b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:23.168554 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d8b5be-1fb6-4cb6-ae11-12996d4d641b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:23.170130 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d8b5be-1fb6-4cb6-ae11-12996d4d641b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:23.189428 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-94d8b5be-1fb6-4cb6-ae11-12996d4d641b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:52:23.189428 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:23.189428 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:23.189428 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:23.189428 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:23.189428 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:23.189428 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:23.189428 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:23.189428 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:23.189428 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:23.189428 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:23.189428 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:23.189428 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:52:23.189428 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:52:23.189428 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:52:23.191192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:52:23.191192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:23.191192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:23.191192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:23.191192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:23.191192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:23.191192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:23.191192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:23.191192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:23.191192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:23.191192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:23.191192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:23.191192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:23.191192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:23.191192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:23.191192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:23.191192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:23.192938 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:52:23.192938 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:52:23.192938 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:52:23.192938 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:23.192938 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 521/1025] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:52:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:23.196834 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998ab01-dd7d-4328-99a7-84e451f19977 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:23.320038 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:23.332415 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998ab01-dd7d-4328-99a7-84e451f19977 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:23.368073 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c25d0-b5d7-43fc-9fdf-48b0da41f92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:23.368658 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c25d0-b5d7-43fc-9fdf-48b0da41f92c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:23.368944 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c25d0-b5d7-43fc-9fdf-48b0da41f92c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:23.369224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c25d0-b5d7-43fc-9fdf-48b0da41f92c None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:23.370766 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c25d0-b5d7-43fc-9fdf-48b0da41f92c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:23.370766 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c25d0-b5d7-43fc-9fdf-48b0da41f92c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:23.371677 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c25d0-b5d7-43fc-9fdf-48b0da41f92c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:23.389621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 505/1026] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:23.391399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8b5be-1fb6-4cb6-ae11-12996d4d641b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:23.426709 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:23.524986 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8b5be-1fb6-4cb6-ae11-12996d4d641b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:23.566513 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8dd4eb-17e6-4c00-9343-d255cb41a87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:23.567453 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8dd4eb-17e6-4c00-9343-d255cb41a87a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:23.567740 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8dd4eb-17e6-4c00-9343-d255cb41a87a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:23.568019 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8dd4eb-17e6-4c00-9343-d255cb41a87a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:23.577067 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8dd4eb-17e6-4c00-9343-d255cb41a87a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:23.578578 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8dd4eb-17e6-4c00-9343-d255cb41a87a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:23.595501 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-3b8dd4eb-17e6-4c00-9343-d255cb41a87a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 15:52:23.595501 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:23.595501 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:23.595501 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:23.595501 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:23.595501 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:23.595501 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:23.595501 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:23.595501 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:23.595501 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:23.595501 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:23.595501 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:23.595501 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:52:23.595501 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:52:23.595501 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:52:23.595501 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:52:23.596564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:23.596564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:23.596564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:52:23.596564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:52:23.596564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:52:23.596564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:52:23.596564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:23.596564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:23.596564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:23.596564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:23.596564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:23.596564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:23.596564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:23.596564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:23.596564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:23.596564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:23.596564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:23.596564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:23.597887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:23.597887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:23.597887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:52:23.597887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:52:23.597887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:52:23.597887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:52:23.597887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:52:23.597887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:52:23.597887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 15:52:23.597887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:23.599090 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 522/1027] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jul 21 15:52:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:52:23.602837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c25d0-b5d7-43fc-9fdf-48b0da41f92c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:23.664863 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bf0ca-436c-4a5d-bc03-49efe5bf1f61 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:23.665060 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bf0ca-436c-4a5d-bc03-49efe5bf1f61 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:23.665324 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bf0ca-436c-4a5d-bc03-49efe5bf1f61 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:23.666858 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 523/1028] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 15:52:23.671747 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092c0be-9375-405d-bf24-15e69bfb01fe None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:23.672050 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092c0be-9375-405d-bf24-15e69bfb01fe None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:23.672334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092c0be-9375-405d-bf24-15e69bfb01fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:23.694421 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a092c0be-9375-405d-bf24-15e69bfb01fe None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:23.695687 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a092c0be-9375-405d-bf24-15e69bfb01fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:23.743208 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c25d0-b5d7-43fc-9fdf-48b0da41f92c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:23.781900 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 524/1029] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 15:52:23.782883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009cfb39-4cf4-4fe0-ad94-ee6cae467cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:23.783555 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cfb39-4cf4-4fe0-ad94-ee6cae467cfd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:23.783838 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cfb39-4cf4-4fe0-ad94-ee6cae467cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:23.784118 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cfb39-4cf4-4fe0-ad94-ee6cae467cfd None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:23.784961 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009cfb39-4cf4-4fe0-ad94-ee6cae467cfd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:23.785278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009cfb39-4cf4-4fe0-ad94-ee6cae467cfd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:23.786756 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009cfb39-4cf4-4fe0-ad94-ee6cae467cfd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:23.800107 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a092c0be-9375-405d-bf24-15e69bfb01fe None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:23.806470 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 506/1030] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jul 21 15:52:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:23.812709 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009cfb39-4cf4-4fe0-ad94-ee6cae467cfd None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:23.825099 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:23.835947 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:23.936274 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009cfb39-4cf4-4fe0-ad94-ee6cae467cfd None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:23.942662 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a092c0be-9375-405d-bf24-15e69bfb01fe None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:23.978244 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec264d-bb17-47f7-9d84-e9fca05c0e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:23.978904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec264d-bb17-47f7-9d84-e9fca05c0e00 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:23.979195 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec264d-bb17-47f7-9d84-e9fca05c0e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:23.979476 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec264d-bb17-47f7-9d84-e9fca05c0e00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:23.989369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa56f44d-563f-4085-9655-41f929828f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:23.990197 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56f44d-563f-4085-9655-41f929828f20 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:23.990486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56f44d-563f-4085-9655-41f929828f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:23.990763 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56f44d-563f-4085-9655-41f929828f20 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:23.991930 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa56f44d-563f-4085-9655-41f929828f20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:23.993382 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa56f44d-563f-4085-9655-41f929828f20 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:23.997825 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 525/1031] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:52:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:52:24.001258 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa56f44d-563f-4085-9655-41f929828f20 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:24.056986 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ec264d-bb17-47f7-9d84-e9fca05c0e00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:24.058491 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ec264d-bb17-47f7-9d84-e9fca05c0e00 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:24.070428 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:24.130153 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:24.141150 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa56f44d-563f-4085-9655-41f929828f20 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:24.179283 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 507/1032] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:52:23 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 15:52:24.185466 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec264d-bb17-47f7-9d84-e9fca05c0e00 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:24.186018 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104331ad-2067-4754-9fd8-a377f86982cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:24.186774 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104331ad-2067-4754-9fd8-a377f86982cb None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:24.187027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104331ad-2067-4754-9fd8-a377f86982cb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:24.187285 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104331ad-2067-4754-9fd8-a377f86982cb None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:24.188327 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104331ad-2067-4754-9fd8-a377f86982cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:24.190703 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104331ad-2067-4754-9fd8-a377f86982cb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:24.213075 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 526/1033] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 15:52:24 2022] POST /identity/v3/users => generated 285 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:52:24.311158 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec264d-bb17-47f7-9d84-e9fca05c0e00 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:24.324105 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:24.362125 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd422f5-3382-4e65-b5cd-a322b0d0d94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:24.362938 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd422f5-3382-4e65-b5cd-a322b0d0d94a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:24.363220 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd422f5-3382-4e65-b5cd-a322b0d0d94a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:24.363496 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd422f5-3382-4e65-b5cd-a322b0d0d94a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:24.377448 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd422f5-3382-4e65-b5cd-a322b0d0d94a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:24.379125 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd422f5-3382-4e65-b5cd-a322b0d0d94a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:24.398991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-1bd422f5-3382-4e65-b5cd-a322b0d0d94a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:52:24.398991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:24.398991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:24.398991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:24.398991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:24.398991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:24.398991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:24.398991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:24.398991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:24.398991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:24.398991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:24.398991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:24.398991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:52:24.398991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:52:24.398991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:52:24.400060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:52:24.400060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:24.400060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:24.400060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:24.400060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:24.400060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:24.400060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:24.400060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:24.400060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:24.400060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:24.400060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:24.400060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:24.400060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:24.400060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:24.400060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:24.400060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:24.400060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:24.400995 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:52:24.400995 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:52:24.400995 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:52:24.400995 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:24.400995 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 508/1034] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:52:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:24.408349 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104331ad-2067-4754-9fd8-a377f86982cb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:24.524179 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104331ad-2067-4754-9fd8-a377f86982cb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:24.560562 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da362371-08f1-45d0-a223-efa2f7a077fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:24.561233 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da362371-08f1-45d0-a223-efa2f7a077fe None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:24.561504 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da362371-08f1-45d0-a223-efa2f7a077fe None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:24.561801 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da362371-08f1-45d0-a223-efa2f7a077fe None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:24.562538 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da362371-08f1-45d0-a223-efa2f7a077fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:24.562846 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da362371-08f1-45d0-a223-efa2f7a077fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:24.564371 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da362371-08f1-45d0-a223-efa2f7a077fe None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:24.577169 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 527/1035] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:52:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:24.582665 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd422f5-3382-4e65-b5cd-a322b0d0d94a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:24.690834 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd422f5-3382-4e65-b5cd-a322b0d0d94a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:24.725756 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d67d31-53e8-4ee3-9415-eb5adfda06ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:24.726557 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d67d31-53e8-4ee3-9415-eb5adfda06ee None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:24.726843 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d67d31-53e8-4ee3-9415-eb5adfda06ee None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:24.727119 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d67d31-53e8-4ee3-9415-eb5adfda06ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:24.734619 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d67d31-53e8-4ee3-9415-eb5adfda06ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:24.736097 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d67d31-53e8-4ee3-9415-eb5adfda06ee None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:24.750720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-34d67d31-53e8-4ee3-9415-eb5adfda06ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:52:24.750720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:24.750720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:24.750720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:24.750720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:24.750720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:24.750720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:24.750720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:24.750720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:24.750720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:24.750720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:24.750720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:24.750720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:52:24.750720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:52:24.750720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:52:24.751691 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:52:24.751691 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:24.751691 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:24.751691 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:24.751691 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:24.751691 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:24.751691 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:24.751691 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:24.751691 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:24.751691 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:24.751691 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:24.751691 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:24.751691 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:24.751691 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:24.751691 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:24.751691 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:24.751691 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:24.752632 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:52:24.752632 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:52:24.752632 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:52:24.752632 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:52:24.752632 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:52:24.752632 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:52:24.752632 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:52:24.752632 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:24.752632 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 509/1036] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:52:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:24.759369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da362371-08f1-45d0-a223-efa2f7a077fe None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:24.771790 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:24.872951 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da362371-08f1-45d0-a223-efa2f7a077fe None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:24.908483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2eb080-c41a-445a-9015-ccf7710ecaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:24.909144 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2eb080-c41a-445a-9015-ccf7710ecaf0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:24.909429 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2eb080-c41a-445a-9015-ccf7710ecaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:24.909737 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2eb080-c41a-445a-9015-ccf7710ecaf0 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:24.910595 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2eb080-c41a-445a-9015-ccf7710ecaf0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:24.910916 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2eb080-c41a-445a-9015-ccf7710ecaf0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:24.912362 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2eb080-c41a-445a-9015-ccf7710ecaf0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:24.925259 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 528/1037] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:52:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:24.931101 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d67d31-53e8-4ee3-9415-eb5adfda06ee None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:24.955705 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:25.039829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9733b9f-ef88-403e-b096-0323391f2945 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:25.040116 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9733b9f-ef88-403e-b096-0323391f2945 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:25.040401 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9733b9f-ef88-403e-b096-0323391f2945 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:25.041972 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 529/1038] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:52:25.047421 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c355f5e-d0e1-4806-8a9e-8f5d77b4a909 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:25.047717 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c355f5e-d0e1-4806-8a9e-8f5d77b4a909 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:25.048052 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c355f5e-d0e1-4806-8a9e-8f5d77b4a909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:25.053592 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d67d31-53e8-4ee3-9415-eb5adfda06ee None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:25.067269 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8c355f5e-d0e1-4806-8a9e-8f5d77b4a909 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:25.068468 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8c355f5e-d0e1-4806-8a9e-8f5d77b4a909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:25.101272 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4e20e3-a570-4b50-a450-05b0b2ae0738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:25.101965 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e20e3-a570-4b50-a450-05b0b2ae0738 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:25.102249 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e20e3-a570-4b50-a450-05b0b2ae0738 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:25.102527 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e20e3-a570-4b50-a450-05b0b2ae0738 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:25.103372 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4e20e3-a570-4b50-a450-05b0b2ae0738 None admin] RBAC: Adding query filter params (role.id=abd0095f0bf64a0c8e694dbf50d6fd9a, scope.project.id=bb1ac00308714c39a79aae09fbb73021, user.id=092553a86f8848dba8c7271f5c0b2c46) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:25.103684 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4e20e3-a570-4b50-a450-05b0b2ae0738 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:25.105126 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4e20e3-a570-4b50-a450-05b0b2ae0738 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:25.113888 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 510/1039] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:52:24 2022] GET /identity/v3/role_assignments?user.id=092553a86f8848dba8c7271f5c0b2c46&scope.project.id=bb1ac00308714c39a79aae09fbb73021&role.id=abd0095f0bf64a0c8e694dbf50d6fd9a => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:25.155067 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:25.169620 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 530/1040] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 15:52:25.191646 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4e20e3-a570-4b50-a450-05b0b2ae0738 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:25.296595 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:25.322980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4e20e3-a570-4b50-a450-05b0b2ae0738 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:25.363276 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b579ed2-1112-4570-8b1f-06697b746954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:25.364072 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b579ed2-1112-4570-8b1f-06697b746954 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:25.364356 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b579ed2-1112-4570-8b1f-06697b746954 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:25.364634 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b579ed2-1112-4570-8b1f-06697b746954 None admin] PATH_INFO: `/v3/services/image` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:25.365289 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b579ed2-1112-4570-8b1f-06697b746954 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:25.366705 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b579ed2-1112-4570-8b1f-06697b746954 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:25.379435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-4b579ed2-1112-4570-8b1f-06697b746954 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 15:52:25.379435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:25.379435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 15:52:25.379435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 15:52:25.379435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 15:52:25.379435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 15:52:25.379435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 15:52:25.379435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:52:25.379435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 15:52:25.379435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:25.379435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 15:52:25.379435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:25.379435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:25.379435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:25.379435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:25.379435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:25.379435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:25.380520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:25.380520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:25.380520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:25.380520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:25.380520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:25.380520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:25.380520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 15:52:25.380520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 15:52:25.380520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 15:52:25.380520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 15:52:25.380520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:25.380520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:25.380520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:25.380520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:25.380520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:25.380520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:25.386376 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:25.386376 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:25.386376 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:25.386376 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:25.386376 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:25.386376 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:25.386376 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:25.386376 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:25.386376 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:25.386376 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:25.386376 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 15:52:25.386376 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:52:25.386376 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 15:52:25.386376 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:25.386376 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 511/1041] 10.222.0.52 () {56 vars in 1105 bytes} [Thu Jul 21 15:52:25 2022] GET /identity/v3/services/image => generated 86 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:25.389252 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c355f5e-d0e1-4806-8a9e-8f5d77b4a909 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:25.500402 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c355f5e-d0e1-4806-8a9e-8f5d77b4a909 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:25.540601 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02822346-7905-448a-88a7-bdb2c6873a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:25.541271 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02822346-7905-448a-88a7-bdb2c6873a3c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:25.541556 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02822346-7905-448a-88a7-bdb2c6873a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:25.541861 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02822346-7905-448a-88a7-bdb2c6873a3c None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:25.542571 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02822346-7905-448a-88a7-bdb2c6873a3c None admin] RBAC: Adding query filter params (name=image) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:25.542879 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02822346-7905-448a-88a7-bdb2c6873a3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:25.544285 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02822346-7905-448a-88a7-bdb2c6873a3c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:25.554066 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 531/1042] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:52:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:25.555178 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b579ed2-1112-4570-8b1f-06697b746954 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:25.658795 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b579ed2-1112-4570-8b1f-06697b746954 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:25.692980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e18a0-76f0-45eb-a476-293e6b07a0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:25.693755 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e18a0-76f0-45eb-a476-293e6b07a0f8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:25.693948 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e18a0-76f0-45eb-a476-293e6b07a0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:25.694229 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e18a0-76f0-45eb-a476-293e6b07a0f8 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:25.694938 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9e18a0-76f0-45eb-a476-293e6b07a0f8 None admin] RBAC: Adding query filter params (type=image) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:25.695250 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9e18a0-76f0-45eb-a476-293e6b07a0f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:25.696621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9e18a0-76f0-45eb-a476-293e6b07a0f8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:25.701794 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 512/1043] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:52:25 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:25.707686 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02822346-7905-448a-88a7-bdb2c6873a3c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:25.769212 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:25.813666 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02822346-7905-448a-88a7-bdb2c6873a3c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:25.847970 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2827950-4508-4b8c-b8e1-28ab0ec8a9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:25.848636 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2827950-4508-4b8c-b8e1-28ab0ec8a9b6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:25.848920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2827950-4508-4b8c-b8e1-28ab0ec8a9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:25.849195 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2827950-4508-4b8c-b8e1-28ab0ec8a9b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:25.850020 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2827950-4508-4b8c-b8e1-28ab0ec8a9b6 None admin] RBAC: Adding query filter params (interface=public, service_id=d398f93604b44b168cde6e4195f64761, region_id=RegionOne) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:25.850337 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2827950-4508-4b8c-b8e1-28ab0ec8a9b6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:25.851718 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2827950-4508-4b8c-b8e1-28ab0ec8a9b6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:25.857558 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 532/1044] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Jul 21 15:52:25 2022] GET /identity/v3/endpoints?service_id=d398f93604b44b168cde6e4195f64761&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:25.863192 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e18a0-76f0-45eb-a476-293e6b07a0f8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:25.879402 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:25.970276 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e18a0-76f0-45eb-a476-293e6b07a0f8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:26.004500 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21919d2f-fde9-434d-ab00-b6094a9ee8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:26.005165 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21919d2f-fde9-434d-ab00-b6094a9ee8a7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:26.005450 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21919d2f-fde9-434d-ab00-b6094a9ee8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:26.005761 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21919d2f-fde9-434d-ab00-b6094a9ee8a7 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:26.006457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21919d2f-fde9-434d-ab00-b6094a9ee8a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:26.007824 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21919d2f-fde9-434d-ab00-b6094a9ee8a7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:26.021686 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 513/1045] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jul 21 15:52:25 2022] GET /identity/v3/services => generated 1957 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 15:52:26.364131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda015b-0d2d-42eb-8d90-131866e0e4f8 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:26.364458 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda015b-0d2d-42eb-8d90-131866e0e4f8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:26.364624 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda015b-0d2d-42eb-8d90-131866e0e4f8 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:26.366256 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 533/1046] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:52:26.371353 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f77bac-f37f-4cd0-9b81-77f2417a299b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:26.371649 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f77bac-f37f-4cd0-9b81-77f2417a299b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:26.371932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f77bac-f37f-4cd0-9b81-77f2417a299b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:26.389101 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f4f77bac-f37f-4cd0-9b81-77f2417a299b None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:26.390335 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f4f77bac-f37f-4cd0-9b81-77f2417a299b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:26.463358 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 514/1047] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:52:26.488439 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda015b-0d2d-42eb-8d90-131866e0e4f8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:26.598546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda015b-0d2d-42eb-8d90-131866e0e4f8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:26.623720 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:26.635101 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d4ce4-ef98-44f9-a4b6-b86559a2d72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:26.635896 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d4ce4-ef98-44f9-a4b6-b86559a2d72a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:26.636182 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d4ce4-ef98-44f9-a4b6-b86559a2d72a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:26.636460 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d4ce4-ef98-44f9-a4b6-b86559a2d72a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:26.646969 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d4ce4-ef98-44f9-a4b6-b86559a2d72a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:52:26.654761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-f26d4ce4-ef98-44f9-a4b6-b86559a2d72a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:52:26.654761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:26.654761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:26.654761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:26.654761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:26.654761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:26.654761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:26.654761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:26.654761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:26.654761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:26.654761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:26.654761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:26.654761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:52:26.654761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:52:26.654761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:52:26.654761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:52:26.655724 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:52:26.655724 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:52:26.655724 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:52:26.655724 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:52:26.655724 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:26.655724 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:26.655724 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:26.655724 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:26.655724 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:26.655724 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:26.655724 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:26.655724 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:26.655724 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:26.655724 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:26.655724 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:26.655724 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:26.656753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:26.656753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:26.656753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:26.656753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:26.656753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:52:26.656753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:52:26.656753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:52:26.656753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:52:26.656753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:52:26.656753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:52:26.656753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:52:26.656753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:26.664057 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 534/1048] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:52:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:26.664209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f77bac-f37f-4cd0-9b81-77f2417a299b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:26.671383 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:26.776767 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f77bac-f37f-4cd0-9b81-77f2417a299b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:26.812460 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e438933-aa3c-49c4-be39-689af73326d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:26.813132 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e438933-aa3c-49c4-be39-689af73326d5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:26.813425 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e438933-aa3c-49c4-be39-689af73326d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:26.813739 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e438933-aa3c-49c4-be39-689af73326d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:26.814507 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e438933-aa3c-49c4-be39-689af73326d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:26.814801 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e438933-aa3c-49c4-be39-689af73326d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:26.816176 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e438933-aa3c-49c4-be39-689af73326d5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:26.829514 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 515/1049] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:52:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:26.831129 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d4ce4-ef98-44f9-a4b6-b86559a2d72a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:26.944038 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d4ce4-ef98-44f9-a4b6-b86559a2d72a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:26.978381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ed018e-3952-4381-a125-a34a4cda5388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:26.979335 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed018e-3952-4381-a125-a34a4cda5388 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:26.979620 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed018e-3952-4381-a125-a34a4cda5388 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:26.979894 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed018e-3952-4381-a125-a34a4cda5388 None admin] PATH_INFO: `/v3/users/1d4cdff81bcf4f7d81c7caeeb518faea` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:26.993178 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ed018e-3952-4381-a125-a34a4cda5388 None admin] RBAC: Authorizing `identity:get_user(user_id=1d4cdff81bcf4f7d81c7caeeb518faea)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:26.994664 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ed018e-3952-4381-a125-a34a4cda5388 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:27.001695 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 535/1050] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:52:26 2022] GET /identity/v3/users/1d4cdff81bcf4f7d81c7caeeb518faea => generated 285 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:27.002659 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e438933-aa3c-49c4-be39-689af73326d5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:27.086008 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:27.122846 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e438933-aa3c-49c4-be39-689af73326d5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:27.169540 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa956b-5fbd-4a47-bccf-22f89a087265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:27.170299 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa956b-5fbd-4a47-bccf-22f89a087265 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:27.170638 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa956b-5fbd-4a47-bccf-22f89a087265 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:27.170927 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa956b-5fbd-4a47-bccf-22f89a087265 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:27.174625 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd5d8a-8a83-44ab-9d3f-2ab3f89de6ba None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:27.174891 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd5d8a-8a83-44ab-9d3f-2ab3f89de6ba None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:27.175174 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd5d8a-8a83-44ab-9d3f-2ab3f89de6ba None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:27.176534 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 536/1051] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:52:27.181569 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3ea5d-478d-4350-8912-034f3c88db61 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:27.181883 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3ea5d-478d-4350-8912-034f3c88db61 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:27.182163 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3ea5d-478d-4350-8912-034f3c88db61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:27.202433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-89d3ea5d-478d-4350-8912-034f3c88db61 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:27.204109 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-89d3ea5d-478d-4350-8912-034f3c88db61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:27.246932 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:27.265875 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa956b-5fbd-4a47-bccf-22f89a087265 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:27.267358 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa956b-5fbd-4a47-bccf-22f89a087265 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:27.284531 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 537/1052] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:27 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 15:52:27.309368 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3ea5d-478d-4350-8912-034f3c88db61 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:27.335188 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:27.371624 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 516/1053] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:52:27 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 15:52:27.377346 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa956b-5fbd-4a47-bccf-22f89a087265 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:27.416807 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3ea5d-478d-4350-8912-034f3c88db61 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:27.451141 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d268341-bae1-4784-9567-33f0397cd2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:27.451981 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d268341-bae1-4784-9567-33f0397cd2e7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:27.452265 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d268341-bae1-4784-9567-33f0397cd2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:27.452592 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d268341-bae1-4784-9567-33f0397cd2e7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:27.463451 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d268341-bae1-4784-9567-33f0397cd2e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:52:27.478388 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-6d268341-bae1-4784-9567-33f0397cd2e7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 15:52:27.478388 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:27.478388 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:27.478388 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:27.478388 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:27.478388 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:27.478388 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:27.478388 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:27.478388 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:27.478388 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:27.478388 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:27.478388 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:27.478388 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:52:27.478388 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:52:27.478388 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:52:27.478388 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:52:27.480380 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:52:27.480380 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:52:27.480380 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:52:27.480380 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:52:27.480380 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:27.480380 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:27.480380 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:27.480380 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:27.480380 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:27.480380 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:27.480380 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:27.480380 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:27.480380 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:27.480380 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:27.480380 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:27.480380 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:27.481389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:27.481389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:27.481389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:27.481389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:27.481389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:52:27.481389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:52:27.481389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:52:27.481389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:52:27.481389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:52:27.481389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:52:27.481389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 15:52:27.481389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:27.481389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 538/1054] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Jul 21 15:52:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:52:27.485969 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d268341-bae1-4784-9567-33f0397cd2e7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:27.487582 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa956b-5fbd-4a47-bccf-22f89a087265 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:27.530726 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:27.536833 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe34828-db46-4006-976e-3cf58472f13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:27.538307 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe34828-db46-4006-976e-3cf58472f13c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:27.538799 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe34828-db46-4006-976e-3cf58472f13c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:27.539125 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe34828-db46-4006-976e-3cf58472f13c None admin] PATH_INFO: `/v3/projects/ec18172a2d984dd4a363a3d91221e557` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:27.540560 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe34828-db46-4006-976e-3cf58472f13c None admin] RBAC: Authorizing `identity:get_project(project_id=ec18172a2d984dd4a363a3d91221e557)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:27.542193 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe34828-db46-4006-976e-3cf58472f13c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:27.544697 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 517/1055] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:52:27 2022] GET /identity/v3/projects/ec18172a2d984dd4a363a3d91221e557 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:27.550644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe34828-db46-4006-976e-3cf58472f13c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:27.589469 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:27.618337 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d268341-bae1-4784-9567-33f0397cd2e7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:27.660542 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947b67d-e899-42d9-beb3-16a2bc16bbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:27.661324 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947b67d-e899-42d9-beb3-16a2bc16bbba None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:27.661708 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947b67d-e899-42d9-beb3-16a2bc16bbba None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:27.662070 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947b67d-e899-42d9-beb3-16a2bc16bbba None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:27.662898 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e947b67d-e899-42d9-beb3-16a2bc16bbba None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:27.663279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e947b67d-e899-42d9-beb3-16a2bc16bbba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:27.664751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e947b67d-e899-42d9-beb3-16a2bc16bbba None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:27.670776 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe34828-db46-4006-976e-3cf58472f13c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:27.673924 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 539/1056] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Jul 21 15:52:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:52:27.679687 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947b67d-e899-42d9-beb3-16a2bc16bbba None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:27.723045 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7a8578-c720-4561-ac6e-bb3226e1bb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:27.723740 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a8578-c720-4561-ac6e-bb3226e1bb86 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:27.724029 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a8578-c720-4561-ac6e-bb3226e1bb86 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:27.724313 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a8578-c720-4561-ac6e-bb3226e1bb86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:27.725158 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7a8578-c720-4561-ac6e-bb3226e1bb86 None admin] RBAC: Adding query filter params (role.id=475521b8a0174037b162f92022aea1a1, scope.project.id=ec18172a2d984dd4a363a3d91221e557, user.id=1d4cdff81bcf4f7d81c7caeeb518faea) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:27.725478 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7a8578-c720-4561-ac6e-bb3226e1bb86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:27.726952 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7a8578-c720-4561-ac6e-bb3226e1bb86 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:27.741612 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 518/1057] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:52:27 2022] GET /identity/v3/role_assignments?user.id=1d4cdff81bcf4f7d81c7caeeb518faea&scope.project.id=ec18172a2d984dd4a363a3d91221e557&role.id=475521b8a0174037b162f92022aea1a1 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:52:27.808798 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:27.818498 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947b67d-e899-42d9-beb3-16a2bc16bbba None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:27.857154 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a162cf35-3f4f-45db-9031-f4411ce934f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:27.858017 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162cf35-3f4f-45db-9031-f4411ce934f2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:27.858299 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162cf35-3f4f-45db-9031-f4411ce934f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:27.858580 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162cf35-3f4f-45db-9031-f4411ce934f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:27.871927 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a162cf35-3f4f-45db-9031-f4411ce934f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:27.874136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a162cf35-3f4f-45db-9031-f4411ce934f2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:27.892862 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-a162cf35-3f4f-45db-9031-f4411ce934f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:52:27.892862 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:27.892862 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:27.892862 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:27.892862 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:27.892862 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:27.892862 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:27.892862 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:27.892862 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:27.892862 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:27.892862 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:27.892862 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:27.892862 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:52:27.892862 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:52:27.892862 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:52:27.893983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:52:27.893983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:27.893983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:27.893983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:27.893983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:27.893983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:27.893983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:27.893983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:27.893983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:27.893983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:27.893983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:27.893983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:27.893983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:27.893983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:27.893983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:27.893983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:27.893983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:27.894921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:52:27.894921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:52:27.894921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:52:27.894921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:27.894921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 540/1058] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:52:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:52:27.900255 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7a8578-c720-4561-ac6e-bb3226e1bb86 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:28.000183 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:28.003820 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7a8578-c720-4561-ac6e-bb3226e1bb86 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:28.040591 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5168e-d6ec-4f0b-8969-a83a1730aa18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:28.041263 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5168e-d6ec-4f0b-8969-a83a1730aa18 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:28.041555 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5168e-d6ec-4f0b-8969-a83a1730aa18 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:28.041865 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5168e-d6ec-4f0b-8969-a83a1730aa18 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:28.042603 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec5168e-d6ec-4f0b-8969-a83a1730aa18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:28.042919 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec5168e-d6ec-4f0b-8969-a83a1730aa18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:28.044826 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec5168e-d6ec-4f0b-8969-a83a1730aa18 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:28.057370 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 519/1059] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:52:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:28.063240 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a162cf35-3f4f-45db-9031-f4411ce934f2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:28.111524 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533e61e-974d-4553-864b-2ac96dca6b4a None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:28.111814 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533e61e-974d-4553-864b-2ac96dca6b4a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:28.112093 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533e61e-974d-4553-864b-2ac96dca6b4a None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:28.113509 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 520/1060] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 15:52:28.120280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075aa41-bae0-4386-bbff-86b86875ef39 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:28.120280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075aa41-bae0-4386-bbff-86b86875ef39 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:28.120280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075aa41-bae0-4386-bbff-86b86875ef39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:28.136073 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e075aa41-bae0-4386-bbff-86b86875ef39 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:28.137257 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e075aa41-bae0-4386-bbff-86b86875ef39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:28.175252 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a162cf35-3f4f-45db-9031-f4411ce934f2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:28.210123 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 521/1061] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:28 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 15:52:28.217200 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9eb109-4349-4d4a-b305-9a15fbf1187f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:28.218018 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9eb109-4349-4d4a-b305-9a15fbf1187f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:28.218298 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9eb109-4349-4d4a-b305-9a15fbf1187f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:28.218656 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9eb109-4349-4d4a-b305-9a15fbf1187f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:28.226883 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9eb109-4349-4d4a-b305-9a15fbf1187f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:28.228331 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9eb109-4349-4d4a-b305-9a15fbf1187f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:28.238708 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e075aa41-bae0-4386-bbff-86b86875ef39 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:28.244867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-bb9eb109-4349-4d4a-b305-9a15fbf1187f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 15:52:28.244867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:28.244867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:28.244867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:28.244867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:28.244867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:28.244867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:28.244867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:28.244867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:28.244867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:28.244867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:28.244867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:28.244867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:52:28.244867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:52:28.244867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:52:28.244867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:52:28.245921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:28.245921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:28.245921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:52:28.245921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:52:28.245921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:52:28.245921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:52:28.245921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:28.245921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:28.245921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:28.245921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:28.245921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:28.245921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:28.245921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:28.245921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:28.245921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:28.245921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:28.245921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:28.245921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:28.248140 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:28.248140 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:28.248140 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:52:28.248140 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:52:28.248140 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:52:28.248140 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:52:28.248140 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:52:28.248140 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:52:28.248140 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 15:52:28.248140 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:28.248140 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 541/1062] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jul 21 15:52:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:28.252409 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9eb109-4349-4d4a-b305-9a15fbf1187f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:28.261433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:28.340789 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:28.352673 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e075aa41-bae0-4386-bbff-86b86875ef39 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:28.359867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9eb109-4349-4d4a-b305-9a15fbf1187f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:28.390197 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2fde65-8ef1-4130-ba67-d12522959c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:28.390992 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2fde65-8ef1-4130-ba67-d12522959c3d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:28.391281 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2fde65-8ef1-4130-ba67-d12522959c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:28.391565 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2fde65-8ef1-4130-ba67-d12522959c3d None admin] PATH_INFO: `/v3/services/image` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:28.392224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2fde65-8ef1-4130-ba67-d12522959c3d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:28.393616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2fde65-8ef1-4130-ba67-d12522959c3d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:28.398967 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9099da-3638-429a-a53b-01521594efb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:28.399696 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9099da-3638-429a-a53b-01521594efb0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:28.399989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9099da-3638-429a-a53b-01521594efb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:28.400270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9099da-3638-429a-a53b-01521594efb0 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:28.401137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9099da-3638-429a-a53b-01521594efb0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:28.401454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9099da-3638-429a-a53b-01521594efb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:28.402932 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9099da-3638-429a-a53b-01521594efb0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:28.405435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-fe2fde65-8ef1-4130-ba67-d12522959c3d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 15:52:28.405435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:28.405435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 15:52:28.405435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 15:52:28.405435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 15:52:28.405435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 15:52:28.405435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 15:52:28.405435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:52:28.405435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 15:52:28.405435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:28.405435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 15:52:28.405435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:28.405435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:28.405435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:28.405435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:28.405435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:28.405435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:28.407027 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:28.407027 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:28.407027 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:28.407027 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:28.407027 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:28.407027 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:28.407027 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 15:52:28.407027 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 15:52:28.407027 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 15:52:28.407027 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 15:52:28.407027 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:28.407027 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:28.407027 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:28.407027 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:28.407027 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:28.407027 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:28.407928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:28.407928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:28.407928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:28.407928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:28.407928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:28.407928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:28.407928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:28.407928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:28.407928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:28.407928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:28.407928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 15:52:28.407928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:52:28.407928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 15:52:28.407928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:28.407928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 522/1063] 10.222.0.52 () {56 vars in 1105 bytes} [Thu Jul 21 15:52:28 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:52:28.414625 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2fde65-8ef1-4130-ba67-d12522959c3d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:28.428781 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 542/1064] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jul 21 15:52:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:52:28.434108 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9099da-3638-429a-a53b-01521594efb0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:28.502234 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:28.534472 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2fde65-8ef1-4130-ba67-d12522959c3d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:28.545233 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9099da-3638-429a-a53b-01521594efb0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:28.569617 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0600c-84b4-4b09-870a-70743986faf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:28.570288 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0600c-84b4-4b09-870a-70743986faf8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:28.570579 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0600c-84b4-4b09-870a-70743986faf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:28.570861 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0600c-84b4-4b09-870a-70743986faf8 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:28.571579 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf0600c-84b4-4b09-870a-70743986faf8 None admin] RBAC: Adding query filter params (name=image) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:28.571899 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf0600c-84b4-4b09-870a-70743986faf8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:28.573269 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf0600c-84b4-4b09-870a-70743986faf8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:28.578863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 523/1065] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:52:28 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:28.583254 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c51280f-9bb6-4e7f-831e-e8f290920089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:28.584049 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51280f-9bb6-4e7f-831e-e8f290920089 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:28.584333 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51280f-9bb6-4e7f-831e-e8f290920089 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:28.584611 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51280f-9bb6-4e7f-831e-e8f290920089 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:28.584829 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0600c-84b4-4b09-870a-70743986faf8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:28.597457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c51280f-9bb6-4e7f-831e-e8f290920089 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:28.598962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c51280f-9bb6-4e7f-831e-e8f290920089 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:28.604596 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:28.616897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-4c51280f-9bb6-4e7f-831e-e8f290920089 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:52:28.616897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:28.616897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:28.616897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:28.616897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:28.616897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:28.616897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:28.616897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:28.616897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:28.616897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:28.616897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:28.616897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:28.616897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:52:28.616897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:52:28.616897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:52:28.617877 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:52:28.617877 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:28.617877 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:28.617877 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:28.617877 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:28.617877 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:28.617877 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:28.617877 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:28.617877 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:28.617877 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:28.617877 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:28.617877 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:28.617877 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:28.617877 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:28.617877 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:28.617877 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:28.617877 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:28.619504 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:52:28.619504 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:52:28.619504 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:52:28.619504 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:28.619504 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 543/1066] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:52:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:52:28.624362 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c51280f-9bb6-4e7f-831e-e8f290920089 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:28.693971 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0600c-84b4-4b09-870a-70743986faf8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:28.728020 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b0d0f-8d55-474d-a8a2-efa213edcf94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:28.728687 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b0d0f-8d55-474d-a8a2-efa213edcf94 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:28.728976 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b0d0f-8d55-474d-a8a2-efa213edcf94 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:28.729256 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b0d0f-8d55-474d-a8a2-efa213edcf94 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:28.729648 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c51280f-9bb6-4e7f-831e-e8f290920089 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:28.729999 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b0d0f-8d55-474d-a8a2-efa213edcf94 None admin] RBAC: Adding query filter params (type=image) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:28.730311 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b0d0f-8d55-474d-a8a2-efa213edcf94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:28.732360 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b0d0f-8d55-474d-a8a2-efa213edcf94 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:28.742102 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 524/1067] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:52:28 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:28.742877 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b0d0f-8d55-474d-a8a2-efa213edcf94 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:28.753069 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:28.773088 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db302ef9-3eca-4e5c-b97a-d0e5bf427703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:28.773886 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db302ef9-3eca-4e5c-b97a-d0e5bf427703 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:28.774309 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db302ef9-3eca-4e5c-b97a-d0e5bf427703 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:28.774596 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db302ef9-3eca-4e5c-b97a-d0e5bf427703 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:28.775373 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db302ef9-3eca-4e5c-b97a-d0e5bf427703 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:28.775684 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db302ef9-3eca-4e5c-b97a-d0e5bf427703 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:28.777068 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db302ef9-3eca-4e5c-b97a-d0e5bf427703 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:28.794944 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 544/1068] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:52:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:28.796473 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db302ef9-3eca-4e5c-b97a-d0e5bf427703 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:28.828240 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:28.850538 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b0d0f-8d55-474d-a8a2-efa213edcf94 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:28.886588 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07b6d7-b084-469a-9d37-bae1667792d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:28.887244 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07b6d7-b084-469a-9d37-bae1667792d8 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:28.887529 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07b6d7-b084-469a-9d37-bae1667792d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:28.887842 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07b6d7-b084-469a-9d37-bae1667792d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:28.888582 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b07b6d7-b084-469a-9d37-bae1667792d8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:28.889992 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b07b6d7-b084-469a-9d37-bae1667792d8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:28.911485 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 525/1069] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jul 21 15:52:28 2022] POST /identity/v3/endpoints => generated 327 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 15:52:28.914865 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db302ef9-3eca-4e5c-b97a-d0e5bf427703 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:28.961429 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33698b11-e763-477c-a4e6-acd49018270a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:28.962294 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33698b11-e763-477c-a4e6-acd49018270a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:28.962579 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33698b11-e763-477c-a4e6-acd49018270a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:28.962858 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33698b11-e763-477c-a4e6-acd49018270a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:28.971171 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33698b11-e763-477c-a4e6-acd49018270a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:28.972920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33698b11-e763-477c-a4e6-acd49018270a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:28.989013 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-33698b11-e763-477c-a4e6-acd49018270a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:52:28.989013 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:28.989013 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:28.989013 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:28.989013 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:28.989013 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:28.989013 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:28.989013 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:28.989013 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:28.989013 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:28.989013 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:28.989013 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:28.989013 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:52:28.989013 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:52:28.989013 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:52:28.990403 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:52:28.990403 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:28.990403 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:28.990403 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:28.990403 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:28.990403 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:28.990403 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:28.990403 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:28.990403 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:28.990403 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:28.990403 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:28.990403 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:28.990403 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:28.990403 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:28.990403 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:28.990403 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:28.990403 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:28.991905 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:52:28.991905 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:52:28.991905 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:52:28.991905 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:52:28.991905 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:52:28.991905 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:52:28.991905 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:52:28.991905 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:28.991905 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 545/1070] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:52:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:52:28.996087 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07b6d7-b084-469a-9d37-bae1667792d8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:29.106555 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07b6d7-b084-469a-9d37-bae1667792d8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:29.142523 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fd652-1c76-41e4-acc5-b9928755c6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:29.143273 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fd652-1c76-41e4-acc5-b9928755c6b2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:29.143624 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fd652-1c76-41e4-acc5-b9928755c6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:29.143967 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fd652-1c76-41e4-acc5-b9928755c6b2 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:29.144834 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932fd652-1c76-41e4-acc5-b9928755c6b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:29.145216 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932fd652-1c76-41e4-acc5-b9928755c6b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:29.146804 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932fd652-1c76-41e4-acc5-b9928755c6b2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:29.154225 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:29.160894 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 526/1071] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:52:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:29.167390 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33698b11-e763-477c-a4e6-acd49018270a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:29.202792 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:29.298946 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33698b11-e763-477c-a4e6-acd49018270a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:29.336910 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5389b-d8ff-46fc-b59b-357a64e4933c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:29.337828 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5389b-d8ff-46fc-b59b-357a64e4933c None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:29.338109 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5389b-d8ff-46fc-b59b-357a64e4933c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:29.338379 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5389b-d8ff-46fc-b59b-357a64e4933c None admin] PATH_INFO: `/v3/projects/bb1ac00308714c39a79aae09fbb73021/users/092553a86f8848dba8c7271f5c0b2c46/roles/abd0095f0bf64a0c8e694dbf50d6fd9a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:29.340558 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea5389b-d8ff-46fc-b59b-357a64e4933c None admin] RBAC: Authorizing `identity:create_grant(project_id=bb1ac00308714c39a79aae09fbb73021, user_id=092553a86f8848dba8c7271f5c0b2c46, role_id=abd0095f0bf64a0c8e694dbf50d6fd9a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:29.342044 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea5389b-d8ff-46fc-b59b-357a64e4933c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:29.368215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 546/1072] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 15:52:29 2022] PUT /identity/v3/projects/bb1ac00308714c39a79aae09fbb73021/users/092553a86f8848dba8c7271f5c0b2c46/roles/abd0095f0bf64a0c8e694dbf50d6fd9a => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 15:52:29.770970 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90de94c-12dc-46ea-a446-6fe548dca62c None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:29.771346 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90de94c-12dc-46ea-a446-6fe548dca62c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:29.771546 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90de94c-12dc-46ea-a446-6fe548dca62c None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:29.772948 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 527/1073] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:52:29.778066 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6a0e1-5d90-4a0e-ad49-1326d46ae11e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:29.778876 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6a0e1-5d90-4a0e-ad49-1326d46ae11e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:29.779151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6a0e1-5d90-4a0e-ad49-1326d46ae11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:29.797439 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-cfb6a0e1-5d90-4a0e-ad49-1326d46ae11e None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:29.798640 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cfb6a0e1-5d90-4a0e-ad49-1326d46ae11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:29.870209 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 547/1074] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:52:29.891750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90de94c-12dc-46ea-a446-6fe548dca62c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:30.001919 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90de94c-12dc-46ea-a446-6fe548dca62c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:30.050879 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213d49d-9cb3-4429-bd05-044448cc335e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:30.051752 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213d49d-9cb3-4429-bd05-044448cc335e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:30.052133 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213d49d-9cb3-4429-bd05-044448cc335e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:30.052505 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213d49d-9cb3-4429-bd05-044448cc335e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:30.064421 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b213d49d-9cb3-4429-bd05-044448cc335e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:52:30.072498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b213d49d-9cb3-4429-bd05-044448cc335e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:52:30.072498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:30.072498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:30.072498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:30.072498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:30.072498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:30.072498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:30.072498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:30.072498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:30.072498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:30.072498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:30.072498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:30.072498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:52:30.072498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:52:30.072498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:52:30.072498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:52:30.073812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:52:30.073812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:52:30.073812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:52:30.073812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:52:30.073812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:30.073812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:30.073812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:30.073812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:30.073812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:30.073812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:30.073812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:30.073812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:30.073812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:30.073812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:30.073812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:30.073812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:30.074910 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:30.074910 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:30.074910 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:30.074910 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:30.074910 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:52:30.074910 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:52:30.074910 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:52:30.074910 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:52:30.074910 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:52:30.074910 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:52:30.074910 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:52:30.074910 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:30.076887 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 528/1075] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:52:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:30.082797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6a0e1-5d90-4a0e-ad49-1326d46ae11e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:30.110485 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:30.195752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6a0e1-5d90-4a0e-ad49-1326d46ae11e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:30.230283 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01d4fe-2152-46bb-bb84-6dc7a7e30c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:30.231067 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01d4fe-2152-46bb-bb84-6dc7a7e30c1b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:30.231481 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01d4fe-2152-46bb-bb84-6dc7a7e30c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:30.233756 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01d4fe-2152-46bb-bb84-6dc7a7e30c1b None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:30.233756 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01d4fe-2152-46bb-bb84-6dc7a7e30c1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:30.233756 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01d4fe-2152-46bb-bb84-6dc7a7e30c1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:30.237023 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01d4fe-2152-46bb-bb84-6dc7a7e30c1b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:30.249106 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 548/1076] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:52:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:30.254858 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213d49d-9cb3-4429-bd05-044448cc335e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:30.313414 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:30.356124 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213d49d-9cb3-4429-bd05-044448cc335e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:30.389795 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b0c19-ab6f-4bcd-8741-6f4f802b527e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:30.390907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b0c19-ab6f-4bcd-8741-6f4f802b527e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:30.391295 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b0c19-ab6f-4bcd-8741-6f4f802b527e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:30.391635 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b0c19-ab6f-4bcd-8741-6f4f802b527e None admin] PATH_INFO: `/v3/users/1d4cdff81bcf4f7d81c7caeeb518faea` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:30.392892 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b0c19-ab6f-4bcd-8741-6f4f802b527e None admin] RBAC: Authorizing `identity:get_user(user_id=1d4cdff81bcf4f7d81c7caeeb518faea)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:30.394427 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b0c19-ab6f-4bcd-8741-6f4f802b527e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:30.396314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 529/1077] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:52:30 2022] GET /identity/v3/users/1d4cdff81bcf4f7d81c7caeeb518faea => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:30.402249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01d4fe-2152-46bb-bb84-6dc7a7e30c1b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:30.492434 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:30.510612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01d4fe-2152-46bb-bb84-6dc7a7e30c1b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:30.546553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885532b-8d38-4fb2-a7ee-52dcb8828e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:30.547506 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885532b-8d38-4fb2-a7ee-52dcb8828e3c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:30.547847 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885532b-8d38-4fb2-a7ee-52dcb8828e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:30.548181 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885532b-8d38-4fb2-a7ee-52dcb8828e3c None admin] PATH_INFO: `/v3/projects/ec18172a2d984dd4a363a3d91221e557` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:30.549437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2885532b-8d38-4fb2-a7ee-52dcb8828e3c None admin] RBAC: Authorizing `identity:get_project(project_id=ec18172a2d984dd4a363a3d91221e557)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:30.550965 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2885532b-8d38-4fb2-a7ee-52dcb8828e3c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:30.552912 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 549/1078] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:52:30 2022] GET /identity/v3/projects/ec18172a2d984dd4a363a3d91221e557 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:30.558725 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b0c19-ab6f-4bcd-8741-6f4f802b527e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:30.667472 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b0c19-ab6f-4bcd-8741-6f4f802b527e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:30.701165 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:30.711468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec439c5-3a5b-488b-9773-672d3f7b91b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:30.712434 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec439c5-3a5b-488b-9773-672d3f7b91b3 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:30.712781 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec439c5-3a5b-488b-9773-672d3f7b91b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:30.713124 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec439c5-3a5b-488b-9773-672d3f7b91b3 None admin] PATH_INFO: `/v3/projects/ec18172a2d984dd4a363a3d91221e557/users/1d4cdff81bcf4f7d81c7caeeb518faea/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:30.715165 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec439c5-3a5b-488b-9773-672d3f7b91b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec18172a2d984dd4a363a3d91221e557, user_id=1d4cdff81bcf4f7d81c7caeeb518faea, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:30.717175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec439c5-3a5b-488b-9773-672d3f7b91b3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:30.733400 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 530/1079] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 15:52:30 2022] PUT /identity/v3/projects/ec18172a2d984dd4a363a3d91221e557/users/1d4cdff81bcf4f7d81c7caeeb518faea/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 15:52:31.001261 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eddf5db-2ff5-4dd8-93e4-fd4a56cb2544 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:31.001635 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eddf5db-2ff5-4dd8-93e4-fd4a56cb2544 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:31.001835 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eddf5db-2ff5-4dd8-93e4-fd4a56cb2544 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:31.003332 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 550/1080] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:52:31.008276 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d69ca2-7ed9-4fad-8fac-590267ece85b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:31.008724 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d69ca2-7ed9-4fad-8fac-590267ece85b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:31.008885 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d69ca2-7ed9-4fad-8fac-590267ece85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:31.026679 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-74d69ca2-7ed9-4fad-8fac-590267ece85b None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:31.027863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-74d69ca2-7ed9-4fad-8fac-590267ece85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:31.097376 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 531/1081] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:52:31.118696 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eddf5db-2ff5-4dd8-93e4-fd4a56cb2544 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:31.223012 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eddf5db-2ff5-4dd8-93e4-fd4a56cb2544 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:31.263580 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c199677-63d6-4a87-b6b0-7cfd17a70cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:31.264616 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c199677-63d6-4a87-b6b0-7cfd17a70cac None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:31.265073 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c199677-63d6-4a87-b6b0-7cfd17a70cac None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:31.265609 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c199677-63d6-4a87-b6b0-7cfd17a70cac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:31.287784 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:31.336278 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c199677-63d6-4a87-b6b0-7cfd17a70cac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:31.337828 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c199677-63d6-4a87-b6b0-7cfd17a70cac None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:31.434422 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1862edb-6698-4c91-a3dd-0abbc2b49026 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:31.435277 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1862edb-6698-4c91-a3dd-0abbc2b49026 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:31.435521 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1862edb-6698-4c91-a3dd-0abbc2b49026 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:31.437075 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 532/1082] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:52:31.441753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 551/1083] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:52:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 15:52:31.442051 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864357f-b84a-4395-a28f-1614987a8875 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:31.442342 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864357f-b84a-4395-a28f-1614987a8875 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:31.442503 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c199677-63d6-4a87-b6b0-7cfd17a70cac None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:31.442608 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864357f-b84a-4395-a28f-1614987a8875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:31.459613 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9864357f-b84a-4395-a28f-1614987a8875 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:31.461007 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9864357f-b84a-4395-a28f-1614987a8875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:31.498205 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:31.539561 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 533/1084] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 15:52:31.553984 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c199677-63d6-4a87-b6b0-7cfd17a70cac None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:31.559247 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864357f-b84a-4395-a28f-1614987a8875 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:31.594773 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a82b3-0f8a-4c92-b3c9-96b520ce80cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:31.595573 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a82b3-0f8a-4c92-b3c9-96b520ce80cc None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:31.595865 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a82b3-0f8a-4c92-b3c9-96b520ce80cc None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:31.596147 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a82b3-0f8a-4c92-b3c9-96b520ce80cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:31.607659 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87a82b3-0f8a-4c92-b3c9-96b520ce80cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:31.609215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87a82b3-0f8a-4c92-b3c9-96b520ce80cc None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:31.615342 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:31.626777 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-a87a82b3-0f8a-4c92-b3c9-96b520ce80cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:52:31.626777 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:31.626777 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:31.626777 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:31.626777 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:31.626777 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:31.626777 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:31.626777 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:31.626777 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:31.626777 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:31.626777 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:31.626777 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:31.626777 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:52:31.626777 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:52:31.626777 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:52:31.627728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:52:31.627728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:31.627728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:31.627728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:31.627728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:31.627728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:31.627728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:31.627728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:31.627728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:31.627728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:31.627728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:31.627728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:31.627728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:31.627728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:31.627728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:31.627728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:31.627728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:31.628712 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:52:31.628712 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:52:31.628712 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:52:31.628712 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:31.628712 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 552/1085] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:52:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:31.633935 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a82b3-0f8a-4c92-b3c9-96b520ce80cc None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:31.678063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864357f-b84a-4395-a28f-1614987a8875 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:31.715478 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6748855b-0edc-4ee0-824e-fb0d2d3aac4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:31.716389 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748855b-0edc-4ee0-824e-fb0d2d3aac4e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:31.716717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748855b-0edc-4ee0-824e-fb0d2d3aac4e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:31.717002 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748855b-0edc-4ee0-824e-fb0d2d3aac4e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:31.731074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6748855b-0edc-4ee0-824e-fb0d2d3aac4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:52:31.739944 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-6748855b-0edc-4ee0-824e-fb0d2d3aac4e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 15:52:31.739944 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:31.739944 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:31.739944 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:31.739944 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:31.739944 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:31.739944 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:31.739944 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:31.739944 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:31.739944 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:31.739944 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:31.739944 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:31.739944 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:52:31.739944 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:52:31.739944 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:52:31.739944 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:52:31.741145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:52:31.741145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:52:31.741145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:52:31.741145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:52:31.741145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:31.741145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:31.741145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:31.741145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:31.741145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:31.741145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:31.741145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:31.741145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:31.741145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:31.741145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:31.741145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:31.741145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:31.742061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:31.742061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:31.742061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:31.742061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:31.742061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:52:31.742061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:52:31.742061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:52:31.742061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:52:31.742061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:52:31.742061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:52:31.742061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 15:52:31.742061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:31.742061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 534/1086] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Jul 21 15:52:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:52:31.747696 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6748855b-0edc-4ee0-824e-fb0d2d3aac4e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:31.752118 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a82b3-0f8a-4c92-b3c9-96b520ce80cc None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:31.755524 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:31.793459 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6892086-d85e-4363-8345-c5b91797b4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:31.794597 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6892086-d85e-4363-8345-c5b91797b4ef None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:31.794885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6892086-d85e-4363-8345-c5b91797b4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:31.795162 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6892086-d85e-4363-8345-c5b91797b4ef None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:31.795894 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6892086-d85e-4363-8345-c5b91797b4ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:31.796203 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6892086-d85e-4363-8345-c5b91797b4ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:31.797640 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6892086-d85e-4363-8345-c5b91797b4ef None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:31.813621 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 553/1087] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:52:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:52:31.892538 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6748855b-0edc-4ee0-824e-fb0d2d3aac4e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:31.909149 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:31.954456 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ccb7d7-e2f0-4510-9efc-2a00120b8468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:31.955151 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccb7d7-e2f0-4510-9efc-2a00120b8468 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:31.955437 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccb7d7-e2f0-4510-9efc-2a00120b8468 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:31.955715 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccb7d7-e2f0-4510-9efc-2a00120b8468 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:31.956486 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ccb7d7-e2f0-4510-9efc-2a00120b8468 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:31.956797 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ccb7d7-e2f0-4510-9efc-2a00120b8468 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:31.958224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ccb7d7-e2f0-4510-9efc-2a00120b8468 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:31.967930 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 535/1088] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Jul 21 15:52:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:52:31.974106 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6892086-d85e-4363-8345-c5b91797b4ef None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:32.083484 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6892086-d85e-4363-8345-c5b91797b4ef None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:32.119791 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c94edc-23d0-492e-8ba7-906d73d7eb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:32.120590 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c94edc-23d0-492e-8ba7-906d73d7eb63 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:32.120878 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c94edc-23d0-492e-8ba7-906d73d7eb63 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:32.121157 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c94edc-23d0-492e-8ba7-906d73d7eb63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:32.133124 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c94edc-23d0-492e-8ba7-906d73d7eb63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:32.134608 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c94edc-23d0-492e-8ba7-906d73d7eb63 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:32.151788 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-91c94edc-23d0-492e-8ba7-906d73d7eb63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:52:32.151788 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:32.151788 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:32.151788 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:32.151788 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:32.151788 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:32.151788 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:32.151788 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:32.151788 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:32.151788 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:32.151788 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:32.151788 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:32.151788 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:52:32.151788 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:52:32.151788 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:52:32.152732 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:52:32.152732 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:32.152732 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:32.152732 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:32.152732 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:32.152732 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:32.152732 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:32.152732 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:32.152732 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:32.152732 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:32.152732 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:32.152732 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:32.152732 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:32.152732 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:32.152732 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:32.152732 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:32.152732 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:32.154366 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:52:32.154366 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:52:32.154366 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:52:32.154366 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:32.154366 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 554/1089] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:52:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:32.159342 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ccb7d7-e2f0-4510-9efc-2a00120b8468 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:32.265263 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ccb7d7-e2f0-4510-9efc-2a00120b8468 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:32.300879 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d48bfa-581e-4856-afbb-cafa94d75998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:32.302103 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d48bfa-581e-4856-afbb-cafa94d75998 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:32.302542 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d48bfa-581e-4856-afbb-cafa94d75998 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:32.302829 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d48bfa-581e-4856-afbb-cafa94d75998 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:32.303576 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d48bfa-581e-4856-afbb-cafa94d75998 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:32.303888 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d48bfa-581e-4856-afbb-cafa94d75998 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:32.306013 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d48bfa-581e-4856-afbb-cafa94d75998 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:32.318982 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 536/1090] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:52:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:32.324591 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c94edc-23d0-492e-8ba7-906d73d7eb63 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:32.345076 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:32.428640 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c94edc-23d0-492e-8ba7-906d73d7eb63 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:32.462504 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec41704-5f1b-4c38-8320-0fbb4b7a5799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:32.463294 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec41704-5f1b-4c38-8320-0fbb4b7a5799 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:32.463574 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec41704-5f1b-4c38-8320-0fbb4b7a5799 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:32.463848 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec41704-5f1b-4c38-8320-0fbb4b7a5799 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:32.472790 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec41704-5f1b-4c38-8320-0fbb4b7a5799 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:32.474279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec41704-5f1b-4c38-8320-0fbb4b7a5799 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:32.489317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-3ec41704-5f1b-4c38-8320-0fbb4b7a5799 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 15:52:32.489317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:32.489317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:32.489317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:32.489317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:32.489317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:32.489317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:32.489317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:32.489317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:32.489317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:32.489317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:32.489317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:32.489317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:52:32.489317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:52:32.489317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:52:32.489317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:52:32.490454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:32.490454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:32.490454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:52:32.490454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:52:32.490454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:52:32.490454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:52:32.490454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:32.490454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:32.490454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:32.490454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:32.490454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:32.490454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:32.490454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:32.490454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:32.490454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:32.490454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:32.490454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:32.490454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:32.491437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:32.491437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:32.491437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:52:32.491437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:52:32.491437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:52:32.491437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:52:32.491437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:52:32.491437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:52:32.491437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 15:52:32.491437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:32.491437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 555/1091] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jul 21 15:52:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:32.496330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d48bfa-581e-4856-afbb-cafa94d75998 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:32.524531 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:32.597738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d48bfa-581e-4856-afbb-cafa94d75998 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:32.632030 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89024dd-1437-4c68-b38a-5bae5db654ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:32.632697 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89024dd-1437-4c68-b38a-5bae5db654ed None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:32.632984 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89024dd-1437-4c68-b38a-5bae5db654ed None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:32.633260 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89024dd-1437-4c68-b38a-5bae5db654ed None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:32.634136 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89024dd-1437-4c68-b38a-5bae5db654ed None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:32.634449 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89024dd-1437-4c68-b38a-5bae5db654ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:32.635899 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89024dd-1437-4c68-b38a-5bae5db654ed None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:32.656044 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 537/1092] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jul 21 15:52:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:32.661842 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec41704-5f1b-4c38-8320-0fbb4b7a5799 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:32.750192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:32.772999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec41704-5f1b-4c38-8320-0fbb4b7a5799 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:32.802767 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89434c6d-584a-45a9-a45b-6a7d0827c927 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:32.803053 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89434c6d-584a-45a9-a45b-6a7d0827c927 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:32.803336 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89434c6d-584a-45a9-a45b-6a7d0827c927 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:32.804819 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 538/1093] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 15:52:32.808271 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc06bcdd-a4a4-4ea5-89df-39f4aa17b8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:32.808928 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06bcdd-a4a4-4ea5-89df-39f4aa17b8d8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:32.809222 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06bcdd-a4a4-4ea5-89df-39f4aa17b8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:32.809530 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06bcdd-a4a4-4ea5-89df-39f4aa17b8d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:32.810053 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3411bc-1306-45ba-80f4-a7151d8c9889 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:32.810339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3411bc-1306-45ba-80f4-a7151d8c9889 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:32.810723 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3411bc-1306-45ba-80f4-a7151d8c9889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:32.836728 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-bd3411bc-1306-45ba-80f4-a7151d8c9889 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:32.837975 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bd3411bc-1306-45ba-80f4-a7151d8c9889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:32.880005 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc06bcdd-a4a4-4ea5-89df-39f4aa17b8d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:32.881389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc06bcdd-a4a4-4ea5-89df-39f4aa17b8d8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:32.905858 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 539/1094] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 15:52:32.929693 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3411bc-1306-45ba-80f4-a7151d8c9889 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:32.949534 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:32.958144 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:32.986927 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 556/1095] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:52:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 15:52:32.992588 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc06bcdd-a4a4-4ea5-89df-39f4aa17b8d8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:33.041093 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3411bc-1306-45ba-80f4-a7151d8c9889 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:33.074904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523dd2ae-dcbb-49d5-9286-99a030c93966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:33.075865 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523dd2ae-dcbb-49d5-9286-99a030c93966 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:33.076155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523dd2ae-dcbb-49d5-9286-99a030c93966 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:33.076432 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523dd2ae-dcbb-49d5-9286-99a030c93966 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:33.086616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523dd2ae-dcbb-49d5-9286-99a030c93966 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:52:33.090449 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc06bcdd-a4a4-4ea5-89df-39f4aa17b8d8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:33.094690 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-523dd2ae-dcbb-49d5-9286-99a030c93966 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:52:33.094690 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:33.094690 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:33.094690 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:33.094690 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:33.094690 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:33.094690 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:33.094690 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:33.094690 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:33.094690 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:33.094690 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:33.094690 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:33.094690 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:52:33.094690 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:52:33.094690 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:52:33.094690 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:52:33.095652 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:52:33.095652 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:52:33.095652 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:52:33.095652 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:52:33.095652 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:33.095652 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:33.095652 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:33.095652 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:33.095652 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:33.095652 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:33.095652 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:33.095652 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:33.095652 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:33.095652 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:33.095652 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:33.095652 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:33.096978 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:33.096978 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:33.096978 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:33.096978 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:33.096978 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:52:33.096978 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:52:33.096978 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:52:33.096978 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:52:33.096978 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:52:33.096978 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:52:33.096978 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:52:33.096978 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:33.096978 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 540/1096] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:52:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:52:33.102089 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523dd2ae-dcbb-49d5-9286-99a030c93966 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:33.124957 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ba375-6559-4c91-8a06-6a53b1e3c4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:33.125774 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ba375-6559-4c91-8a06-6a53b1e3c4bc None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:33.126050 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ba375-6559-4c91-8a06-6a53b1e3c4bc None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:33.126328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ba375-6559-4c91-8a06-6a53b1e3c4bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:33.136780 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928ba375-6559-4c91-8a06-6a53b1e3c4bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:33.138714 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928ba375-6559-4c91-8a06-6a53b1e3c4bc None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:33.157439 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-928ba375-6559-4c91-8a06-6a53b1e3c4bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:52:33.157439 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:33.157439 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:33.157439 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:33.157439 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:33.157439 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:33.157439 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:33.157439 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:33.157439 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:33.157439 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:33.157439 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:33.157439 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:33.157439 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:52:33.157439 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:52:33.157439 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:52:33.158442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:52:33.158442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:33.158442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:33.158442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:33.158442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:33.158442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:33.158442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:33.158442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:33.158442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:33.158442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:33.158442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:33.158442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:33.158442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:33.158442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:33.158442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:33.158442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:33.158442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:33.159523 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:52:33.159523 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:52:33.159523 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:52:33.159523 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:33.159523 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 557/1097] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:52:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:52:33.164190 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ba375-6559-4c91-8a06-6a53b1e3c4bc None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:33.181786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:33.192846 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:33.206334 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523dd2ae-dcbb-49d5-9286-99a030c93966 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:33.243494 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb26b95-24fb-42aa-92da-d21bc382b6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:33.244165 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb26b95-24fb-42aa-92da-d21bc382b6cd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:33.244456 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb26b95-24fb-42aa-92da-d21bc382b6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:33.244737 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb26b95-24fb-42aa-92da-d21bc382b6cd None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:33.245504 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb26b95-24fb-42aa-92da-d21bc382b6cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:33.245850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb26b95-24fb-42aa-92da-d21bc382b6cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:33.247232 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb26b95-24fb-42aa-92da-d21bc382b6cd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:33.255592 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 541/1098] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:52:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:52:33.261409 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb26b95-24fb-42aa-92da-d21bc382b6cd None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:33.269280 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ba375-6559-4c91-8a06-6a53b1e3c4bc None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:33.303833 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b39ba2-88a9-4196-a87b-3f24a1ec3d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:33.304494 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b39ba2-88a9-4196-a87b-3f24a1ec3d49 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:33.304789 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b39ba2-88a9-4196-a87b-3f24a1ec3d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:33.305066 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b39ba2-88a9-4196-a87b-3f24a1ec3d49 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:33.305822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b39ba2-88a9-4196-a87b-3f24a1ec3d49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:33.306130 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b39ba2-88a9-4196-a87b-3f24a1ec3d49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:33.307505 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b39ba2-88a9-4196-a87b-3f24a1ec3d49 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:33.318758 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 558/1099] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:52:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:52:33.324333 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b39ba2-88a9-4196-a87b-3f24a1ec3d49 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:33.361628 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb26b95-24fb-42aa-92da-d21bc382b6cd None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:33.396050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fadf04-a4e7-435d-84d6-56042ba949a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:33.396839 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fadf04-a4e7-435d-84d6-56042ba949a2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:33.397127 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fadf04-a4e7-435d-84d6-56042ba949a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:33.397406 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fadf04-a4e7-435d-84d6-56042ba949a2 None admin] PATH_INFO: `/v3/users/1d4cdff81bcf4f7d81c7caeeb518faea` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:33.398643 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fadf04-a4e7-435d-84d6-56042ba949a2 None admin] RBAC: Authorizing `identity:get_user(user_id=1d4cdff81bcf4f7d81c7caeeb518faea)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:33.400109 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fadf04-a4e7-435d-84d6-56042ba949a2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:33.401929 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 542/1100] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:52:33 2022] GET /identity/v3/users/1d4cdff81bcf4f7d81c7caeeb518faea => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:52:33.409207 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fadf04-a4e7-435d-84d6-56042ba949a2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:33.419015 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:33.420777 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:33.433677 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b39ba2-88a9-4196-a87b-3f24a1ec3d49 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:33.469058 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205d05f-17f3-481f-b388-befc2e51f915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:33.469891 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205d05f-17f3-481f-b388-befc2e51f915 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:33.470178 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205d05f-17f3-481f-b388-befc2e51f915 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:33.470456 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205d05f-17f3-481f-b388-befc2e51f915 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:33.477629 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0205d05f-17f3-481f-b388-befc2e51f915 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:33.479097 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0205d05f-17f3-481f-b388-befc2e51f915 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:33.493342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-0205d05f-17f3-481f-b388-befc2e51f915 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:52:33.493342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:33.493342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:33.493342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:33.493342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:33.493342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:33.493342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:33.493342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:33.493342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:33.493342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:33.493342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:33.493342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:33.493342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:52:33.493342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:52:33.493342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:52:33.494399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:52:33.494399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:33.494399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:33.494399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:33.494399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:33.494399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:33.494399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:33.494399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:33.494399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:33.494399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:33.494399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:33.494399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:33.494399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:33.494399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:33.494399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:33.494399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:33.494399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:33.495354 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:52:33.495354 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:52:33.495354 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:52:33.495354 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:52:33.495354 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:52:33.495354 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:52:33.495354 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:52:33.495354 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:33.495354 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 559/1101] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:52:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:33.501326 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205d05f-17f3-481f-b388-befc2e51f915 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:33.515466 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fadf04-a4e7-435d-84d6-56042ba949a2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:33.548962 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e2948-650b-45c3-804e-ee9c1c5057fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:33.549636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e2948-650b-45c3-804e-ee9c1c5057fc None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:33.549919 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e2948-650b-45c3-804e-ee9c1c5057fc None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:33.550196 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e2948-650b-45c3-804e-ee9c1c5057fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:33.601288 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205d05f-17f3-481f-b388-befc2e51f915 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:33.623234 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e2948-650b-45c3-804e-ee9c1c5057fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:33.624781 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e2948-650b-45c3-804e-ee9c1c5057fc None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:33.632778 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:33.652025 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f2f87-c7db-400d-84e4-e0f93f028123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:33.652025 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f2f87-c7db-400d-84e4-e0f93f028123 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:33.652025 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f2f87-c7db-400d-84e4-e0f93f028123 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:33.652025 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f2f87-c7db-400d-84e4-e0f93f028123 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:33.652493 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881f2f87-c7db-400d-84e4-e0f93f028123 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:33.652493 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881f2f87-c7db-400d-84e4-e0f93f028123 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:33.652493 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881f2f87-c7db-400d-84e4-e0f93f028123 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:33.665232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 560/1102] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:52:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:52:33.666223 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f2f87-c7db-400d-84e4-e0f93f028123 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:33.677412 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:33.738439 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 543/1103] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:52:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 15:52:33.739545 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e2948-650b-45c3-804e-ee9c1c5057fc None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:33.768352 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f2f87-c7db-400d-84e4-e0f93f028123 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:33.801779 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a0c05f-d5c5-4cd0-bc72-2327b29ce06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:33.802440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a0c05f-d5c5-4cd0-bc72-2327b29ce06a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:33.802731 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a0c05f-d5c5-4cd0-bc72-2327b29ce06a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:33.803013 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a0c05f-d5c5-4cd0-bc72-2327b29ce06a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:33.803849 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a0c05f-d5c5-4cd0-bc72-2327b29ce06a None admin] RBAC: Adding query filter params (role.id=abd0095f0bf64a0c8e694dbf50d6fd9a, scope.project.id=bb1ac00308714c39a79aae09fbb73021, user.id=092553a86f8848dba8c7271f5c0b2c46) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:33.804168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a0c05f-d5c5-4cd0-bc72-2327b29ce06a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:33.805638 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a0c05f-d5c5-4cd0-bc72-2327b29ce06a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:33.811975 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 561/1104] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:52:33 2022] GET /identity/v3/role_assignments?user.id=092553a86f8848dba8c7271f5c0b2c46&scope.project.id=bb1ac00308714c39a79aae09fbb73021&role.id=abd0095f0bf64a0c8e694dbf50d6fd9a => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:52:33.841942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e2948-650b-45c3-804e-ee9c1c5057fc None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:33.850531 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:33.881278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3bb79-02ed-4c2a-a82d-b98aaee6b1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:33.882193 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3bb79-02ed-4c2a-a82d-b98aaee6b1db None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:33.882477 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3bb79-02ed-4c2a-a82d-b98aaee6b1db None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:33.882754 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3bb79-02ed-4c2a-a82d-b98aaee6b1db None admin] PATH_INFO: `/v3/projects/ec18172a2d984dd4a363a3d91221e557` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:33.884001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb3bb79-02ed-4c2a-a82d-b98aaee6b1db None admin] RBAC: Authorizing `identity:get_project(project_id=ec18172a2d984dd4a363a3d91221e557)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:33.885467 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb3bb79-02ed-4c2a-a82d-b98aaee6b1db None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:33.887264 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 544/1105] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:52:33 2022] GET /identity/v3/projects/ec18172a2d984dd4a363a3d91221e557 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:33.893642 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a0c05f-d5c5-4cd0-bc72-2327b29ce06a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:33.980739 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:34.016044 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a0c05f-d5c5-4cd0-bc72-2327b29ce06a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:34.052937 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0d91c-5b63-4d28-a312-c4c778f87974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:34.053619 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0d91c-5b63-4d28-a312-c4c778f87974 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:34.053904 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0d91c-5b63-4d28-a312-c4c778f87974 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:34.054179 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0d91c-5b63-4d28-a312-c4c778f87974 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:34.055023 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a0d91c-5b63-4d28-a312-c4c778f87974 None admin] RBAC: Adding query filter params (role.id=475521b8a0174037b162f92022aea1a1, scope.project.id=ec18172a2d984dd4a363a3d91221e557, user.id=1d4cdff81bcf4f7d81c7caeeb518faea) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:34.055339 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a0d91c-5b63-4d28-a312-c4c778f87974 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:34.056783 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a0d91c-5b63-4d28-a312-c4c778f87974 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:34.064045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 562/1106] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:52:33 2022] GET /identity/v3/role_assignments?user.id=1d4cdff81bcf4f7d81c7caeeb518faea&scope.project.id=ec18172a2d984dd4a363a3d91221e557&role.id=475521b8a0174037b162f92022aea1a1 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:36.161963 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767880a2-d3d0-4604-acee-ccbebbdf1c0d None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:36.162533 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767880a2-d3d0-4604-acee-ccbebbdf1c0d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:36.162533 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767880a2-d3d0-4604-acee-ccbebbdf1c0d None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:36.163889 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 545/1107] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:52:36.169195 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352591a6-d28d-41e4-a500-eb5ff4edeb50 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:36.169651 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352591a6-d28d-41e4-a500-eb5ff4edeb50 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:36.169786 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352591a6-d28d-41e4-a500-eb5ff4edeb50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:36.186951 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-352591a6-d28d-41e4-a500-eb5ff4edeb50 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:36.188146 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-352591a6-d28d-41e4-a500-eb5ff4edeb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:36.260466 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 563/1108] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:52:36.285926 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767880a2-d3d0-4604-acee-ccbebbdf1c0d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:36.387892 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767880a2-d3d0-4604-acee-ccbebbdf1c0d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:36.426318 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeee6d0-c58c-4d10-8da9-5d238e330f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:36.427019 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeee6d0-c58c-4d10-8da9-5d238e330f72 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:36.427307 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeee6d0-c58c-4d10-8da9-5d238e330f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:36.427588 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeee6d0-c58c-4d10-8da9-5d238e330f72 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:36.428829 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeee6d0-c58c-4d10-8da9-5d238e330f72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:36.430310 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeee6d0-c58c-4d10-8da9-5d238e330f72 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:36.431964 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 546/1109] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:52:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:36.437760 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352591a6-d28d-41e4-a500-eb5ff4edeb50 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:36.500938 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:36.533800 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352591a6-d28d-41e4-a500-eb5ff4edeb50 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:36.566224 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f01b3-e69c-43f7-969e-01ac0da04aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:36.566872 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f01b3-e69c-43f7-969e-01ac0da04aa3 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:36.567151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f01b3-e69c-43f7-969e-01ac0da04aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:36.567426 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f01b3-e69c-43f7-969e-01ac0da04aa3 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:36.568131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95f01b3-e69c-43f7-969e-01ac0da04aa3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:36.569558 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95f01b3-e69c-43f7-969e-01ac0da04aa3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:36.589175 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 564/1110] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 15:52:36 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:52:38.672666 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc444d2d-e5f3-44d0-8808-10aeeda013b3 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:38.673174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc444d2d-e5f3-44d0-8808-10aeeda013b3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:38.673174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc444d2d-e5f3-44d0-8808-10aeeda013b3 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:38.674540 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 547/1111] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:52:38.679728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ac233-2ac5-4526-aa77-8faaa36cf5d5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:38.680135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ac233-2ac5-4526-aa77-8faaa36cf5d5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:38.680274 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ac233-2ac5-4526-aa77-8faaa36cf5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:38.697551 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-126ac233-2ac5-4526-aa77-8faaa36cf5d5 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:38.698763 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-126ac233-2ac5-4526-aa77-8faaa36cf5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:38.770036 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 565/1112] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:52:38.794526 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc444d2d-e5f3-44d0-8808-10aeeda013b3 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:38.839388 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:38.903701 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc444d2d-e5f3-44d0-8808-10aeeda013b3 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:38.943009 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d3132-c00f-48ea-afce-05d2f162e0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:38.943237 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d3132-c00f-48ea-afce-05d2f162e0b2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:38.943341 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d3132-c00f-48ea-afce-05d2f162e0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:38.943635 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d3132-c00f-48ea-afce-05d2f162e0b2 None admin] PATH_INFO: `/v3/roles/85dcd5509fca4911881d45baf5f1d5a5` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:38.945156 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d3132-c00f-48ea-afce-05d2f162e0b2 None admin] RBAC: Authorizing `identity:get_role(role_id=85dcd5509fca4911881d45baf5f1d5a5)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:38.946594 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d3132-c00f-48ea-afce-05d2f162e0b2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:38.948042 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 548/1113] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:52:38 2022] GET /identity/v3/roles/85dcd5509fca4911881d45baf5f1d5a5 => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:38.954691 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ac233-2ac5-4526-aa77-8faaa36cf5d5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:39.009468 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:39.067239 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ac233-2ac5-4526-aa77-8faaa36cf5d5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:39.101678 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d807db29-4012-448d-8aad-eb41c5b15527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:39.102562 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807db29-4012-448d-8aad-eb41c5b15527 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:39.102849 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807db29-4012-448d-8aad-eb41c5b15527 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:39.103128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807db29-4012-448d-8aad-eb41c5b15527 None admin] PATH_INFO: `/v3/users/e968e4a4fba3435ea900514f4edefd22` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:39.116988 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d807db29-4012-448d-8aad-eb41c5b15527 None admin] RBAC: Authorizing `identity:get_user(user_id=e968e4a4fba3435ea900514f4edefd22)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:39.118487 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d807db29-4012-448d-8aad-eb41c5b15527 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:39.120183 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 566/1114] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:52:38 2022] GET /identity/v3/users/e968e4a4fba3435ea900514f4edefd22 => generated 286 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:39.126121 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d3132-c00f-48ea-afce-05d2f162e0b2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:39.226267 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d3132-c00f-48ea-afce-05d2f162e0b2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:39.237239 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:39.260823 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7693cc0-6de7-461c-93f8-204c5321a38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:39.261472 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7693cc0-6de7-461c-93f8-204c5321a38d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:39.261787 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7693cc0-6de7-461c-93f8-204c5321a38d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:39.262068 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7693cc0-6de7-461c-93f8-204c5321a38d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:39.331869 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7693cc0-6de7-461c-93f8-204c5321a38d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:39.333334 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7693cc0-6de7-461c-93f8-204c5321a38d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:39.442659 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 549/1115] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:52:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 15:52:39.449147 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d807db29-4012-448d-8aad-eb41c5b15527 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:39.557596 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d807db29-4012-448d-8aad-eb41c5b15527 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:39.569008 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:39.598031 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7011acc-6d55-4d69-94f2-c0a9fbe901df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:39.598895 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7011acc-6d55-4d69-94f2-c0a9fbe901df None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:39.599174 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7011acc-6d55-4d69-94f2-c0a9fbe901df None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:39.599446 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7011acc-6d55-4d69-94f2-c0a9fbe901df None admin] PATH_INFO: `/v3/projects/ec18172a2d984dd4a363a3d91221e557` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:39.600622 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7011acc-6d55-4d69-94f2-c0a9fbe901df None admin] RBAC: Authorizing `identity:get_project(project_id=ec18172a2d984dd4a363a3d91221e557)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:39.602096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7011acc-6d55-4d69-94f2-c0a9fbe901df None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:39.603798 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 567/1116] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:52:39 2022] GET /identity/v3/projects/ec18172a2d984dd4a363a3d91221e557 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:39.610039 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7693cc0-6de7-461c-93f8-204c5321a38d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:39.623341 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:39.712035 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7693cc0-6de7-461c-93f8-204c5321a38d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:39.746377 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad24ba9-f205-49a0-b9d3-6cbdc6101eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:39.747562 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad24ba9-f205-49a0-b9d3-6cbdc6101eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:39.747847 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad24ba9-f205-49a0-b9d3-6cbdc6101eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:39.748124 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad24ba9-f205-49a0-b9d3-6cbdc6101eb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:39.748961 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad24ba9-f205-49a0-b9d3-6cbdc6101eb7 None admin] RBAC: Adding query filter params (role.id=85dcd5509fca4911881d45baf5f1d5a5, scope.project.id=ec18172a2d984dd4a363a3d91221e557, user.id=e968e4a4fba3435ea900514f4edefd22) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:39.749277 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad24ba9-f205-49a0-b9d3-6cbdc6101eb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:39.750745 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad24ba9-f205-49a0-b9d3-6cbdc6101eb7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:39.756187 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 550/1117] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:52:39 2022] GET /identity/v3/role_assignments?user.id=e968e4a4fba3435ea900514f4edefd22&scope.project.id=ec18172a2d984dd4a363a3d91221e557&role.id=85dcd5509fca4911881d45baf5f1d5a5 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:41.750945 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e12412-3925-4756-ba75-4f7fa28a4d8c None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:41.751513 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e12412-3925-4756-ba75-4f7fa28a4d8c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:41.751513 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e12412-3925-4756-ba75-4f7fa28a4d8c None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:41.752876 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 568/1118] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:52:41.758510 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72eb1d4-0ec3-4494-b63d-54aad81e7a84 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:41.758962 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72eb1d4-0ec3-4494-b63d-54aad81e7a84 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:41.759075 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72eb1d4-0ec3-4494-b63d-54aad81e7a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:41.776397 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a72eb1d4-0ec3-4494-b63d-54aad81e7a84 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:41.777656 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a72eb1d4-0ec3-4494-b63d-54aad81e7a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:41.848267 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 551/1119] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:52:41.871367 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e12412-3925-4756-ba75-4f7fa28a4d8c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:41.976234 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e12412-3925-4756-ba75-4f7fa28a4d8c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:42.010249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d8948-80f4-4011-94e7-41729e18b514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:42.011039 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d8948-80f4-4011-94e7-41729e18b514 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:42.011324 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d8948-80f4-4011-94e7-41729e18b514 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:42.011606 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d8948-80f4-4011-94e7-41729e18b514 None admin] PATH_INFO: `/v3/roles/85dcd5509fca4911881d45baf5f1d5a5` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:42.013664 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16d8948-80f4-4011-94e7-41729e18b514 None admin] RBAC: Authorizing `identity:get_role(role_id=85dcd5509fca4911881d45baf5f1d5a5)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:42.015061 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16d8948-80f4-4011-94e7-41729e18b514 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:42.016445 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 569/1120] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:52:41 2022] GET /identity/v3/roles/85dcd5509fca4911881d45baf5f1d5a5 => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:42.022433 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72eb1d4-0ec3-4494-b63d-54aad81e7a84 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:42.025563 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:42.120697 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72eb1d4-0ec3-4494-b63d-54aad81e7a84 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:42.153980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e9b86a-7d2f-423a-bb42-148b42e803b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:42.154760 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9b86a-7d2f-423a-bb42-148b42e803b0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:42.155043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9b86a-7d2f-423a-bb42-148b42e803b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:42.155317 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9b86a-7d2f-423a-bb42-148b42e803b0 None admin] PATH_INFO: `/v3/users/e968e4a4fba3435ea900514f4edefd22` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:42.156499 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e9b86a-7d2f-423a-bb42-148b42e803b0 None admin] RBAC: Authorizing `identity:get_user(user_id=e968e4a4fba3435ea900514f4edefd22)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:42.157964 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e9b86a-7d2f-423a-bb42-148b42e803b0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:42.159589 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 552/1121] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:52:42 2022] GET /identity/v3/users/e968e4a4fba3435ea900514f4edefd22 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:42.165490 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d8948-80f4-4011-94e7-41729e18b514 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:42.212268 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:42.267416 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d8948-80f4-4011-94e7-41729e18b514 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:42.301798 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668fbf6e-6f61-4242-94a8-2b5dd78aef30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:42.302469 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668fbf6e-6f61-4242-94a8-2b5dd78aef30 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:42.302756 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668fbf6e-6f61-4242-94a8-2b5dd78aef30 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:42.303035 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668fbf6e-6f61-4242-94a8-2b5dd78aef30 None admin] PATH_INFO: `/v3/projects/ec18172a2d984dd4a363a3d91221e557` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:42.304223 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668fbf6e-6f61-4242-94a8-2b5dd78aef30 None admin] RBAC: Authorizing `identity:get_project(project_id=ec18172a2d984dd4a363a3d91221e557)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:42.305688 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668fbf6e-6f61-4242-94a8-2b5dd78aef30 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:42.307386 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 570/1122] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:52:42 2022] GET /identity/v3/projects/ec18172a2d984dd4a363a3d91221e557 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:42.314047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e9b86a-7d2f-423a-bb42-148b42e803b0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:42.397310 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:42.426224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e9b86a-7d2f-423a-bb42-148b42e803b0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:42.458958 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6ddf0-520d-4e9f-9e56-2c45c7dee0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:42.459833 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6ddf0-520d-4e9f-9e56-2c45c7dee0cc None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:42.460114 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6ddf0-520d-4e9f-9e56-2c45c7dee0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:42.460387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6ddf0-520d-4e9f-9e56-2c45c7dee0cc None admin] PATH_INFO: `/v3/projects/ec18172a2d984dd4a363a3d91221e557/users/e968e4a4fba3435ea900514f4edefd22/roles/85dcd5509fca4911881d45baf5f1d5a5` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:42.462562 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6ddf0-520d-4e9f-9e56-2c45c7dee0cc None admin] RBAC: Authorizing `identity:create_grant(project_id=ec18172a2d984dd4a363a3d91221e557, user_id=e968e4a4fba3435ea900514f4edefd22, role_id=85dcd5509fca4911881d45baf5f1d5a5)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:42.463996 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6ddf0-520d-4e9f-9e56-2c45c7dee0cc None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:42.476370 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 553/1123] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 15:52:42 2022] PUT /identity/v3/projects/ec18172a2d984dd4a363a3d91221e557/users/e968e4a4fba3435ea900514f4edefd22/roles/85dcd5509fca4911881d45baf5f1d5a5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 15:52:44.520982 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abf82d-764a-47a1-a9fe-d08217f2aead None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:44.521499 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abf82d-764a-47a1-a9fe-d08217f2aead None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:44.521499 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abf82d-764a-47a1-a9fe-d08217f2aead None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:44.522992 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 571/1124] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:52:44.528182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e6a47-a53c-42db-b0cd-2be1b6671290 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:44.528552 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e6a47-a53c-42db-b0cd-2be1b6671290 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:44.528747 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e6a47-a53c-42db-b0cd-2be1b6671290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:44.545509 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-050e6a47-a53c-42db-b0cd-2be1b6671290 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:44.546740 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-050e6a47-a53c-42db-b0cd-2be1b6671290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:44.613064 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 554/1125] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:52:44.636712 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16abf82d-764a-47a1-a9fe-d08217f2aead None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:44.749113 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16abf82d-764a-47a1-a9fe-d08217f2aead None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:44.765020 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:44.788176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06525c2-4146-4544-8978-21e487d31932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:44.789051 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06525c2-4146-4544-8978-21e487d31932 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:44.789338 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06525c2-4146-4544-8978-21e487d31932 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:44.789642 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06525c2-4146-4544-8978-21e487d31932 None admin] PATH_INFO: `/v3/roles/85dcd5509fca4911881d45baf5f1d5a5` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:44.791178 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06525c2-4146-4544-8978-21e487d31932 None admin] RBAC: Authorizing `identity:get_role(role_id=85dcd5509fca4911881d45baf5f1d5a5)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:44.792556 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06525c2-4146-4544-8978-21e487d31932 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:44.794529 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 572/1126] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:52:44 2022] GET /identity/v3/roles/85dcd5509fca4911881d45baf5f1d5a5 => generated 223 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:44.800857 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e6a47-a53c-42db-b0cd-2be1b6671290 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:44.866611 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:44.913445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e6a47-a53c-42db-b0cd-2be1b6671290 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:44.956862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb90ea1e-a982-4ca1-85fd-dc0cb54e688b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:44.957784 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90ea1e-a982-4ca1-85fd-dc0cb54e688b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:44.958071 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90ea1e-a982-4ca1-85fd-dc0cb54e688b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:44.958350 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90ea1e-a982-4ca1-85fd-dc0cb54e688b None admin] PATH_INFO: `/v3/users/e968e4a4fba3435ea900514f4edefd22` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:44.959540 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb90ea1e-a982-4ca1-85fd-dc0cb54e688b None admin] RBAC: Authorizing `identity:get_user(user_id=e968e4a4fba3435ea900514f4edefd22)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:44.960978 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb90ea1e-a982-4ca1-85fd-dc0cb54e688b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:44.962666 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 555/1127] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:52:44 2022] GET /identity/v3/users/e968e4a4fba3435ea900514f4edefd22 => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:52:44.969352 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06525c2-4146-4544-8978-21e487d31932 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:45.089781 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06525c2-4146-4544-8978-21e487d31932 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:45.124330 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af6887f-2155-40b4-9e43-6e70f88affc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:45.124976 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6887f-2155-40b4-9e43-6e70f88affc9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:45.125253 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6887f-2155-40b4-9e43-6e70f88affc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:45.125525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6887f-2155-40b4-9e43-6e70f88affc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:45.174856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:45.196800 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af6887f-2155-40b4-9e43-6e70f88affc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:45.198882 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af6887f-2155-40b4-9e43-6e70f88affc9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:45.300568 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 573/1128] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:52:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 15:52:45.306645 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb90ea1e-a982-4ca1-85fd-dc0cb54e688b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:45.409181 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb90ea1e-a982-4ca1-85fd-dc0cb54e688b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:45.443688 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:45.450237 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8781a-c092-47b2-bed0-8827161f2c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:45.451096 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8781a-c092-47b2-bed0-8827161f2c09 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:45.451380 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8781a-c092-47b2-bed0-8827161f2c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:45.451656 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8781a-c092-47b2-bed0-8827161f2c09 None admin] PATH_INFO: `/v3/projects/ec18172a2d984dd4a363a3d91221e557` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:45.452837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e8781a-c092-47b2-bed0-8827161f2c09 None admin] RBAC: Authorizing `identity:get_project(project_id=ec18172a2d984dd4a363a3d91221e557)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:45.454311 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e8781a-c092-47b2-bed0-8827161f2c09 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:45.456029 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 556/1129] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:52:45 2022] GET /identity/v3/projects/ec18172a2d984dd4a363a3d91221e557 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:45.462195 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af6887f-2155-40b4-9e43-6e70f88affc9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:45.544532 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:45.563309 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af6887f-2155-40b4-9e43-6e70f88affc9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:45.600222 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbaa207-5a93-4b50-b813-48ba2bb08de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:45.600940 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbaa207-5a93-4b50-b813-48ba2bb08de0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:45.601226 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbaa207-5a93-4b50-b813-48ba2bb08de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:45.601504 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbaa207-5a93-4b50-b813-48ba2bb08de0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:45.602375 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbaa207-5a93-4b50-b813-48ba2bb08de0 None admin] RBAC: Adding query filter params (role.id=85dcd5509fca4911881d45baf5f1d5a5, scope.project.id=ec18172a2d984dd4a363a3d91221e557, user.id=e968e4a4fba3435ea900514f4edefd22) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:45.602694 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbaa207-5a93-4b50-b813-48ba2bb08de0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:45.604147 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbaa207-5a93-4b50-b813-48ba2bb08de0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:45.610624 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 574/1130] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:52:45 2022] GET /identity/v3/role_assignments?user.id=e968e4a4fba3435ea900514f4edefd22&scope.project.id=ec18172a2d984dd4a363a3d91221e557&role.id=85dcd5509fca4911881d45baf5f1d5a5 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:47.666858 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8a5f1-1328-4ed6-9d61-aaa708268e0c None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:47.667365 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8a5f1-1328-4ed6-9d61-aaa708268e0c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:47.667365 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8a5f1-1328-4ed6-9d61-aaa708268e0c None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:47.669173 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 557/1131] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:52:47.674316 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5e15f-fbbc-4947-aa8f-894e3a42506c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:47.674739 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5e15f-fbbc-4947-aa8f-894e3a42506c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:47.674888 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5e15f-fbbc-4947-aa8f-894e3a42506c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:47.691922 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4fe5e15f-fbbc-4947-aa8f-894e3a42506c None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:47.693123 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4fe5e15f-fbbc-4947-aa8f-894e3a42506c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:47.768113 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 575/1132] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:52:47.792289 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8a5f1-1328-4ed6-9d61-aaa708268e0c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:47.892976 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8a5f1-1328-4ed6-9d61-aaa708268e0c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:47.933000 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c0f30-26e3-4e2c-be66-484bba450da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:47.933731 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c0f30-26e3-4e2c-be66-484bba450da8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:47.934015 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c0f30-26e3-4e2c-be66-484bba450da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:47.934354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c0f30-26e3-4e2c-be66-484bba450da8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:47.935542 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c0f30-26e3-4e2c-be66-484bba450da8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:47.936979 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c0f30-26e3-4e2c-be66-484bba450da8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:47.938679 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 558/1133] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:52:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:47.944594 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5e15f-fbbc-4947-aa8f-894e3a42506c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:48.020290 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:48.043881 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5e15f-fbbc-4947-aa8f-894e3a42506c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:48.080137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1fe048-a118-44a2-9198-c3d5bd4dad5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:48.080802 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1fe048-a118-44a2-9198-c3d5bd4dad5c None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:48.081082 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1fe048-a118-44a2-9198-c3d5bd4dad5c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:48.081354 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1fe048-a118-44a2-9198-c3d5bd4dad5c None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:48.082268 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1fe048-a118-44a2-9198-c3d5bd4dad5c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:48.083705 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1fe048-a118-44a2-9198-c3d5bd4dad5c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:48.112920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 576/1134] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 21 15:52:47 2022] POST /identity/v3/projects => generated 311 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:52:50.193163 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71cdc6e-268b-402a-9494-61939d4b32ab None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:50.193787 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71cdc6e-268b-402a-9494-61939d4b32ab None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:50.193787 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71cdc6e-268b-402a-9494-61939d4b32ab None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:50.195128 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 559/1135] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:52:50.200531 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c4ffb3-365f-417e-8502-a23760554ab9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:50.200929 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c4ffb3-365f-417e-8502-a23760554ab9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:50.201096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c4ffb3-365f-417e-8502-a23760554ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:50.217947 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-33c4ffb3-365f-417e-8502-a23760554ab9 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:50.219245 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-33c4ffb3-365f-417e-8502-a23760554ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:50.285475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 577/1136] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:52:50.310024 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71cdc6e-268b-402a-9494-61939d4b32ab None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:50.374602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:50.410042 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71cdc6e-268b-402a-9494-61939d4b32ab None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:50.442145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e36b8-a7d4-4dfd-8c9f-29b7013b39b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:50.442920 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e36b8-a7d4-4dfd-8c9f-29b7013b39b3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:50.443251 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e36b8-a7d4-4dfd-8c9f-29b7013b39b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:50.443476 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e36b8-a7d4-4dfd-8c9f-29b7013b39b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:50.444632 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e36b8-a7d4-4dfd-8c9f-29b7013b39b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:50.446093 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e36b8-a7d4-4dfd-8c9f-29b7013b39b3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:50.447706 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 560/1137] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:52:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:50.454091 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c4ffb3-365f-417e-8502-a23760554ab9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:50.511032 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:50.553311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c4ffb3-365f-417e-8502-a23760554ab9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:50.590669 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a06ee-426f-4791-93b5-df78fb1f6d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:50.591341 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a06ee-426f-4791-93b5-df78fb1f6d25 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:50.591628 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a06ee-426f-4791-93b5-df78fb1f6d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:50.591906 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a06ee-426f-4791-93b5-df78fb1f6d25 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:50.592620 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a06ee-426f-4791-93b5-df78fb1f6d25 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:50.594104 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a06ee-426f-4791-93b5-df78fb1f6d25 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:50.614467 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 578/1138] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 15:52:50 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:52:52.682996 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe963013-d810-4d34-9bcb-828c29cf657d None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:52.683504 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe963013-d810-4d34-9bcb-828c29cf657d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:52.683592 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe963013-d810-4d34-9bcb-828c29cf657d None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:52.684897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 561/1139] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:52:52.690057 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22ed41-e746-4fab-8a73-657fd8f3f976 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:52.690434 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22ed41-e746-4fab-8a73-657fd8f3f976 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:52.690615 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22ed41-e746-4fab-8a73-657fd8f3f976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:52.707688 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0b22ed41-e746-4fab-8a73-657fd8f3f976 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:52.708873 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0b22ed41-e746-4fab-8a73-657fd8f3f976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:52.779590 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 579/1140] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:52:52.803918 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe963013-d810-4d34-9bcb-828c29cf657d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:52.907981 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe963013-d810-4d34-9bcb-828c29cf657d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:52.943173 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb445a-d114-48a3-b245-eec25527a2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:52.943881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb445a-d114-48a3-b245-eec25527a2a5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:52.944165 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb445a-d114-48a3-b245-eec25527a2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:52.944442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb445a-d114-48a3-b245-eec25527a2a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:52.946457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:52.956502 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edb445a-d114-48a3-b245-eec25527a2a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:52:52.964215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-4edb445a-d114-48a3-b245-eec25527a2a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:52:52.964215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:52.964215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:52.964215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:52.964215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:52.964215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:52.964215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:52.964215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:52.964215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:52.964215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:52.964215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:52.964215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:52.964215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:52:52.964215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:52:52.964215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:52:52.964215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:52:52.965260 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:52:52.965260 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:52:52.965260 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:52:52.965260 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:52:52.965260 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:52.965260 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:52.965260 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:52.965260 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:52.965260 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:52.965260 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:52.965260 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:52.965260 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:52.965260 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:52.965260 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:52.965260 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:52.965260 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:52.966199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:52.966199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:52.966199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:52.966199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:52.966199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:52:52.966199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:52:52.966199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:52:52.966199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:52:52.966199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:52:52.966199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:52:52.966199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:52:52.966199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:52.966199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 562/1141] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:52:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:52.971970 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22ed41-e746-4fab-8a73-657fd8f3f976 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:53.045254 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:53.107033 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22ed41-e746-4fab-8a73-657fd8f3f976 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:53.142395 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec487b7f-6c81-41a4-b714-a6f33a5851c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:53.143073 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec487b7f-6c81-41a4-b714-a6f33a5851c9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:53.143362 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec487b7f-6c81-41a4-b714-a6f33a5851c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:53.143642 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec487b7f-6c81-41a4-b714-a6f33a5851c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:53.144402 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec487b7f-6c81-41a4-b714-a6f33a5851c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:53.144718 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec487b7f-6c81-41a4-b714-a6f33a5851c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:53.146153 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec487b7f-6c81-41a4-b714-a6f33a5851c9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:53.155391 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 580/1142] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:52:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:52:53.161360 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb445a-d114-48a3-b245-eec25527a2a5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:53.262016 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb445a-d114-48a3-b245-eec25527a2a5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:53.295752 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b6436-b249-4cfa-a37d-44cee5a49e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:53.296611 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b6436-b249-4cfa-a37d-44cee5a49e9d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:53.296898 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b6436-b249-4cfa-a37d-44cee5a49e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:53.297179 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b6436-b249-4cfa-a37d-44cee5a49e9d None admin] PATH_INFO: `/v3/users/148d893943fc4b36a4432c4023130446` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:53.310779 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b6436-b249-4cfa-a37d-44cee5a49e9d None admin] RBAC: Authorizing `identity:get_user(user_id=148d893943fc4b36a4432c4023130446)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:53.312222 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b6436-b249-4cfa-a37d-44cee5a49e9d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:53.313905 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 563/1143] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:52:53 2022] GET /identity/v3/users/148d893943fc4b36a4432c4023130446 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:53.319809 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec487b7f-6c81-41a4-b714-a6f33a5851c9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:53.419477 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec487b7f-6c81-41a4-b714-a6f33a5851c9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:53.426279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:53.454685 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543078ca-0be8-4448-8049-d06c94bdb56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:53.455329 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543078ca-0be8-4448-8049-d06c94bdb56d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:53.455616 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543078ca-0be8-4448-8049-d06c94bdb56d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:53.455891 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543078ca-0be8-4448-8049-d06c94bdb56d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:53.523802 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543078ca-0be8-4448-8049-d06c94bdb56d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:53.525190 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543078ca-0be8-4448-8049-d06c94bdb56d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:53.620231 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 581/1144] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:52:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 15:52:53.626816 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b6436-b249-4cfa-a37d-44cee5a49e9d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:53.682017 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:53.729022 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b6436-b249-4cfa-a37d-44cee5a49e9d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:53.762827 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fb8b55-02ee-4f8d-b9cb-1bab38eca4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:53.763675 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb8b55-02ee-4f8d-b9cb-1bab38eca4f3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:53.763961 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb8b55-02ee-4f8d-b9cb-1bab38eca4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:53.764238 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb8b55-02ee-4f8d-b9cb-1bab38eca4f3 None admin] PATH_INFO: `/v3/projects/c1802f0c69524e1aa0c7d55fe62740da` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:53.765401 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fb8b55-02ee-4f8d-b9cb-1bab38eca4f3 None admin] RBAC: Authorizing `identity:get_project(project_id=c1802f0c69524e1aa0c7d55fe62740da)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:53.766862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fb8b55-02ee-4f8d-b9cb-1bab38eca4f3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:53.768554 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 564/1145] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:52:53 2022] GET /identity/v3/projects/c1802f0c69524e1aa0c7d55fe62740da => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:53.774838 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543078ca-0be8-4448-8049-d06c94bdb56d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:53.783348 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:53.871511 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543078ca-0be8-4448-8049-d06c94bdb56d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:53.903819 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b3321e-3e22-4c4c-be4d-51e8bddc0d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:53.904453 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3321e-3e22-4c4c-be4d-51e8bddc0d14 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:53.904733 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3321e-3e22-4c4c-be4d-51e8bddc0d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:53.905005 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3321e-3e22-4c4c-be4d-51e8bddc0d14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:53.905853 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b3321e-3e22-4c4c-be4d-51e8bddc0d14 None admin] RBAC: Adding query filter params (role.id=475521b8a0174037b162f92022aea1a1, scope.project.id=c1802f0c69524e1aa0c7d55fe62740da, user.id=148d893943fc4b36a4432c4023130446) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:53.906163 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b3321e-3e22-4c4c-be4d-51e8bddc0d14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:53.907583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b3321e-3e22-4c4c-be4d-51e8bddc0d14 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:53.912851 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 582/1146] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:52:53 2022] GET /identity/v3/role_assignments?user.id=148d893943fc4b36a4432c4023130446&scope.project.id=c1802f0c69524e1aa0c7d55fe62740da&role.id=475521b8a0174037b162f92022aea1a1 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:55.909521 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9747841-8ed7-4a13-a2d4-4a80001ca37d None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:55.910114 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9747841-8ed7-4a13-a2d4-4a80001ca37d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:55.910114 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9747841-8ed7-4a13-a2d4-4a80001ca37d None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:55.911485 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 565/1147] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:52:55.916661 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5d004-2f22-4ae9-adb5-cc0cbc38eeeb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:55.917106 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5d004-2f22-4ae9-adb5-cc0cbc38eeeb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:55.917211 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5d004-2f22-4ae9-adb5-cc0cbc38eeeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:55.934783 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0ab5d004-2f22-4ae9-adb5-cc0cbc38eeeb None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:55.935969 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0ab5d004-2f22-4ae9-adb5-cc0cbc38eeeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:55.998287 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:56.003494 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 583/1148] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:52:56.025887 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9747841-8ed7-4a13-a2d4-4a80001ca37d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:56.126803 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9747841-8ed7-4a13-a2d4-4a80001ca37d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:56.146312 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:56.160463 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcbd07f-98a7-40bf-af13-5020a7f3b93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:56.161186 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcbd07f-98a7-40bf-af13-5020a7f3b93d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:56.161472 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcbd07f-98a7-40bf-af13-5020a7f3b93d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:56.161787 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcbd07f-98a7-40bf-af13-5020a7f3b93d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:56.172170 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcbd07f-98a7-40bf-af13-5020a7f3b93d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:52:56.179777 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-2fcbd07f-98a7-40bf-af13-5020a7f3b93d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:52:56.179777 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:56.179777 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:56.179777 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:56.179777 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:56.179777 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:56.179777 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:56.179777 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:56.179777 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:56.179777 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:56.179777 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:56.179777 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:56.179777 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:52:56.179777 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:52:56.179777 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:52:56.179777 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:52:56.180883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:52:56.180883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:52:56.180883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:52:56.180883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:52:56.180883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:56.180883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:56.180883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:56.180883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:56.180883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:56.180883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:56.180883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:56.180883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:56.180883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:56.180883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:56.180883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:56.180883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:56.181955 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:56.181955 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:56.181955 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:56.181955 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:56.181955 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:52:56.181955 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:52:56.181955 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:52:56.181955 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:52:56.181955 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:52:56.181955 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:52:56.181955 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:52:56.181955 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:52:56.181955 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 566/1149] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:52:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:56.187148 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5d004-2f22-4ae9-adb5-cc0cbc38eeeb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:56.287266 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5d004-2f22-4ae9-adb5-cc0cbc38eeeb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:56.320564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b578509-97ac-4a9c-8136-b024ae405e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:56.321224 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b578509-97ac-4a9c-8136-b024ae405e4f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:56.321513 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b578509-97ac-4a9c-8136-b024ae405e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:56.321819 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b578509-97ac-4a9c-8136-b024ae405e4f None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:56.322573 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b578509-97ac-4a9c-8136-b024ae405e4f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:56.322886 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b578509-97ac-4a9c-8136-b024ae405e4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:56.324297 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b578509-97ac-4a9c-8136-b024ae405e4f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:56.332283 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 584/1150] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:52:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:56.338097 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcbd07f-98a7-40bf-af13-5020a7f3b93d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:56.437266 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcbd07f-98a7-40bf-af13-5020a7f3b93d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:56.470797 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736536ad-6759-4898-b00f-e9b336872949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:56.471928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736536ad-6759-4898-b00f-e9b336872949 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:56.472208 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736536ad-6759-4898-b00f-e9b336872949 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:56.472482 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736536ad-6759-4898-b00f-e9b336872949 None admin] PATH_INFO: `/v3/users/148d893943fc4b36a4432c4023130446` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:56.474126 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736536ad-6759-4898-b00f-e9b336872949 None admin] RBAC: Authorizing `identity:get_user(user_id=148d893943fc4b36a4432c4023130446)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:56.476109 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736536ad-6759-4898-b00f-e9b336872949 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:56.478257 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 567/1151] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:52:56 2022] GET /identity/v3/users/148d893943fc4b36a4432c4023130446 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:56.484618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b578509-97ac-4a9c-8136-b024ae405e4f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:56.552038 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:56.591363 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b578509-97ac-4a9c-8136-b024ae405e4f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:56.625300 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d56dc-fbc9-489c-bc91-30fa812a2511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:56.626186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d56dc-fbc9-489c-bc91-30fa812a2511 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:56.626469 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d56dc-fbc9-489c-bc91-30fa812a2511 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:56.626740 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d56dc-fbc9-489c-bc91-30fa812a2511 None admin] PATH_INFO: `/v3/projects/c1802f0c69524e1aa0c7d55fe62740da` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:56.627910 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d56dc-fbc9-489c-bc91-30fa812a2511 None admin] RBAC: Authorizing `identity:get_project(project_id=c1802f0c69524e1aa0c7d55fe62740da)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:56.629352 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d56dc-fbc9-489c-bc91-30fa812a2511 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:56.631102 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 585/1152] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:52:56 2022] GET /identity/v3/projects/c1802f0c69524e1aa0c7d55fe62740da => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:56.637741 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736536ad-6759-4898-b00f-e9b336872949 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:56.685369 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:56.735623 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736536ad-6759-4898-b00f-e9b336872949 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:56.769239 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d68b119-7cc0-421d-b856-90661fbfea8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:56.770148 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d68b119-7cc0-421d-b856-90661fbfea8b None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:56.770437 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d68b119-7cc0-421d-b856-90661fbfea8b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:56.770718 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d68b119-7cc0-421d-b856-90661fbfea8b None admin] PATH_INFO: `/v3/projects/c1802f0c69524e1aa0c7d55fe62740da/users/148d893943fc4b36a4432c4023130446/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:56.772675 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d68b119-7cc0-421d-b856-90661fbfea8b None admin] RBAC: Authorizing `identity:create_grant(project_id=c1802f0c69524e1aa0c7d55fe62740da, user_id=148d893943fc4b36a4432c4023130446, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:56.774155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d68b119-7cc0-421d-b856-90661fbfea8b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:56.788483 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 568/1153] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 15:52:56 2022] PUT /identity/v3/projects/c1802f0c69524e1aa0c7d55fe62740da/users/148d893943fc4b36a4432c4023130446/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 15:52:58.806497 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93027dc3-17c3-440b-8903-676c7345211b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:58.807090 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93027dc3-17c3-440b-8903-676c7345211b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:58.807090 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93027dc3-17c3-440b-8903-676c7345211b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:58.809107 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 586/1154] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:52:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:52:58.814248 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65fed3-6900-4b9a-9de6-988faf9c0d0c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:58.814671 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65fed3-6900-4b9a-9de6-988faf9c0d0c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:58.814816 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65fed3-6900-4b9a-9de6-988faf9c0d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:58.831365 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-bd65fed3-6900-4b9a-9de6-988faf9c0d0c None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:52:58.832558 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bd65fed3-6900-4b9a-9de6-988faf9c0d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:52:58.900925 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 569/1155] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:52:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:52:58.924565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93027dc3-17c3-440b-8903-676c7345211b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:59.029319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93027dc3-17c3-440b-8903-676c7345211b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:59.057606 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:59.063669 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fad7d0-9c8a-4701-a13c-ec316e8ec0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:59.064451 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fad7d0-9c8a-4701-a13c-ec316e8ec0c2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:59.064730 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fad7d0-9c8a-4701-a13c-ec316e8ec0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:59.064999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fad7d0-9c8a-4701-a13c-ec316e8ec0c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:59.074913 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fad7d0-9c8a-4701-a13c-ec316e8ec0c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:52:59.082483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-62fad7d0-9c8a-4701-a13c-ec316e8ec0c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:52:59.082483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:52:59.082483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:52:59.082483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:52:59.082483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:52:59.082483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:52:59.082483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:52:59.082483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:52:59.082483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:52:59.082483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:52:59.082483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:52:59.082483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:52:59.082483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:52:59.082483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:52:59.082483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:52:59.082483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:52:59.083685 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:52:59.083685 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:52:59.083685 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:52:59.083685 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:52:59.083685 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:52:59.083685 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:52:59.083685 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:52:59.083685 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:52:59.083685 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:52:59.083685 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:52:59.083685 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:52:59.083685 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:52:59.083685 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:52:59.083685 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:52:59.083685 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:52:59.083685 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:52:59.084622 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:52:59.084622 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:52:59.084622 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:52:59.084622 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:52:59.084622 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:52:59.084622 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:52:59.084622 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:52:59.084622 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:52:59.084622 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:52:59.084622 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:52:59.084622 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:52:59.084622 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:52:59.084622 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 587/1156] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:52:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:52:59.090210 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65fed3-6900-4b9a-9de6-988faf9c0d0c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:59.128777 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:59.191754 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65fed3-6900-4b9a-9de6-988faf9c0d0c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:59.227087 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca84881-2f72-450b-923d-dc06ba63778f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:59.227742 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca84881-2f72-450b-923d-dc06ba63778f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:59.228023 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca84881-2f72-450b-923d-dc06ba63778f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:59.228297 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca84881-2f72-450b-923d-dc06ba63778f None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:59.229044 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca84881-2f72-450b-923d-dc06ba63778f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:59.229349 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca84881-2f72-450b-923d-dc06ba63778f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:59.231308 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca84881-2f72-450b-923d-dc06ba63778f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:59.239544 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 570/1157] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:52:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:59.245385 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fad7d0-9c8a-4701-a13c-ec316e8ec0c2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:59.342754 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fad7d0-9c8a-4701-a13c-ec316e8ec0c2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:59.375159 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d90fa9-fa03-495a-85ac-1790e4be3e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:59.376015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d90fa9-fa03-495a-85ac-1790e4be3e04 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:59.376299 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d90fa9-fa03-495a-85ac-1790e4be3e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:59.376576 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d90fa9-fa03-495a-85ac-1790e4be3e04 None admin] PATH_INFO: `/v3/users/148d893943fc4b36a4432c4023130446` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:59.377783 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d90fa9-fa03-495a-85ac-1790e4be3e04 None admin] RBAC: Authorizing `identity:get_user(user_id=148d893943fc4b36a4432c4023130446)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:59.379226 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d90fa9-fa03-495a-85ac-1790e4be3e04 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:59.380859 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 588/1158] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:52:59 2022] GET /identity/v3/users/148d893943fc4b36a4432c4023130446 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:59.387118 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca84881-2f72-450b-923d-dc06ba63778f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:59.481758 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:59.484936 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca84881-2f72-450b-923d-dc06ba63778f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:59.517890 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a0743-d577-4cae-af72-1719b5304bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:59.518528 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a0743-d577-4cae-af72-1719b5304bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:59.518811 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a0743-d577-4cae-af72-1719b5304bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:59.519085 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a0743-d577-4cae-af72-1719b5304bb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:59.584123 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161a0743-d577-4cae-af72-1719b5304bb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:59.585564 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161a0743-d577-4cae-af72-1719b5304bb5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:59.677337 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:59.681170 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 571/1159] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:52:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 15:52:59.687523 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d90fa9-fa03-495a-85ac-1790e4be3e04 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:59.748738 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:52:59.791155 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d90fa9-fa03-495a-85ac-1790e4be3e04 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:59.824412 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d13c5-5a38-45b8-9fb3-143a6c44edab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:59.825267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d13c5-5a38-45b8-9fb3-143a6c44edab None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:59.825550 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d13c5-5a38-45b8-9fb3-143a6c44edab None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:59.825858 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d13c5-5a38-45b8-9fb3-143a6c44edab None admin] PATH_INFO: `/v3/projects/c1802f0c69524e1aa0c7d55fe62740da` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:59.827037 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d13c5-5a38-45b8-9fb3-143a6c44edab None admin] RBAC: Authorizing `identity:get_project(project_id=c1802f0c69524e1aa0c7d55fe62740da)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:59.828478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d13c5-5a38-45b8-9fb3-143a6c44edab None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:59.830205 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 589/1160] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:52:59 2022] GET /identity/v3/projects/c1802f0c69524e1aa0c7d55fe62740da => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:52:59.836319 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a0743-d577-4cae-af72-1719b5304bb5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:52:59.932077 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a0743-d577-4cae-af72-1719b5304bb5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:52:59.965405 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadbc361-a016-4ab0-9f16-fd8a57ff3ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:52:59.966068 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbc361-a016-4ab0-9f16-fd8a57ff3ebc None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:52:59.966357 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbc361-a016-4ab0-9f16-fd8a57ff3ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:52:59.966638 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbc361-a016-4ab0-9f16-fd8a57ff3ebc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:52:59.967468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadbc361-a016-4ab0-9f16-fd8a57ff3ebc None admin] RBAC: Adding query filter params (role.id=475521b8a0174037b162f92022aea1a1, scope.project.id=c1802f0c69524e1aa0c7d55fe62740da, user.id=148d893943fc4b36a4432c4023130446) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:52:59.967784 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadbc361-a016-4ab0-9f16-fd8a57ff3ebc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:52:59.969206 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadbc361-a016-4ab0-9f16-fd8a57ff3ebc None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:52:59.975215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 572/1161] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:52:59 2022] GET /identity/v3/role_assignments?user.id=148d893943fc4b36a4432c4023130446&scope.project.id=c1802f0c69524e1aa0c7d55fe62740da&role.id=475521b8a0174037b162f92022aea1a1 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:53:02.020338 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef7164-2edd-4330-ae9c-b23ace68577f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:02.020912 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef7164-2edd-4330-ae9c-b23ace68577f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:02.020912 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef7164-2edd-4330-ae9c-b23ace68577f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:02.022267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 590/1162] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:53:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:53:02.027720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe948633-c491-4bb4-a9f9-81e2100654d8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:02.028137 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe948633-c491-4bb4-a9f9-81e2100654d8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:02.028290 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe948633-c491-4bb4-a9f9-81e2100654d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:02.044696 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-fe948633-c491-4bb4-a9f9-81e2100654d8 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:53:02.045943 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fe948633-c491-4bb4-a9f9-81e2100654d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:53:02.104491 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:53:02.112981 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 573/1163] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:53:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:53:02.135601 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef7164-2edd-4330-ae9c-b23ace68577f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:53:02.247778 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef7164-2edd-4330-ae9c-b23ace68577f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:53:02.275160 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:53:02.283885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a71d3-125d-47be-a24e-ee8ba91924f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:53:02.284532 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a71d3-125d-47be-a24e-ee8ba91924f3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:02.284812 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a71d3-125d-47be-a24e-ee8ba91924f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:02.285083 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a71d3-125d-47be-a24e-ee8ba91924f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:02.356984 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a71d3-125d-47be-a24e-ee8ba91924f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:53:02.358470 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a71d3-125d-47be-a24e-ee8ba91924f3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:53:02.459287 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 591/1164] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:53:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 15:53:02.465310 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe948633-c491-4bb4-a9f9-81e2100654d8 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:53:02.567187 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe948633-c491-4bb4-a9f9-81e2100654d8 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:53:02.601167 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeda7f0-bcbc-467d-bfa3-7f77c6e61d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:53:02.602004 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeda7f0-bcbc-467d-bfa3-7f77c6e61d77 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:02.602295 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeda7f0-bcbc-467d-bfa3-7f77c6e61d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:02.602578 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeda7f0-bcbc-467d-bfa3-7f77c6e61d77 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:02.609993 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeda7f0-bcbc-467d-bfa3-7f77c6e61d77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:53:02.611475 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeda7f0-bcbc-467d-bfa3-7f77c6e61d77 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:53:02.626519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-5eeda7f0-bcbc-467d-bfa3-7f77c6e61d77 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 21 15:53:02.626519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:53:02.626519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 21 15:53:02.626519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 21 15:53:02.626519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:53:02.626519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:53:02.626519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:53:02.626519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:53:02.626519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 21 15:53:02.626519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:53:02.626519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 15:53:02.626519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:53:02.626519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:53:02.626519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:53:02.626519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:53:02.626519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:53:02.626519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:53:02.627679 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:53:02.627679 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:53:02.627679 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:53:02.627679 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:53:02.627679 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:53:02.627679 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:53:02.627679 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:53:02.627679 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:53:02.627679 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:53:02.627679 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:53:02.627679 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:53:02.627679 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:53:02.627679 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:53:02.627679 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:53:02.627679 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:53:02.627679 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:53:02.628626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:53:02.628626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:53:02.628626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:53:02.628626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:53:02.628626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:53:02.628626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:53:02.628626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:53:02.628626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:53:02.628626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:53:02.628626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:53:02.628626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 21 15:53:02.628626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:53:02.628626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 21 15:53:02.628626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:53:02.628626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 574/1165] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jul 21 15:53:02 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:53:02.633888 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a71d3-125d-47be-a24e-ee8ba91924f3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:53:02.660010 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:53:02.733319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a71d3-125d-47be-a24e-ee8ba91924f3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:53:02.767835 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f5c122-36d6-4e7a-91ed-a2b057da7c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:53:02.768498 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5c122-36d6-4e7a-91ed-a2b057da7c4f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:02.768786 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5c122-36d6-4e7a-91ed-a2b057da7c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:02.769063 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5c122-36d6-4e7a-91ed-a2b057da7c4f None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:02.769815 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f5c122-36d6-4e7a-91ed-a2b057da7c4f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:53:02.770124 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f5c122-36d6-4e7a-91ed-a2b057da7c4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:53:02.771502 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f5c122-36d6-4e7a-91ed-a2b057da7c4f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:53:02.779401 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 592/1166] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Jul 21 15:53:02 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:53:04.827897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee70348-4e29-45df-8ef0-951746aa6584 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:04.828462 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee70348-4e29-45df-8ef0-951746aa6584 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:04.828462 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee70348-4e29-45df-8ef0-951746aa6584 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:04.829874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 575/1167] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:53:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:53:04.835195 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31408e83-de36-466a-a290-06ee6c521a21 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:04.835670 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31408e83-de36-466a-a290-06ee6c521a21 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:04.835766 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31408e83-de36-466a-a290-06ee6c521a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:04.853111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-31408e83-de36-466a-a290-06ee6c521a21 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:53:04.854340 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-31408e83-de36-466a-a290-06ee6c521a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:53:04.936556 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 593/1168] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:53:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 15:53:04.959236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee70348-4e29-45df-8ef0-951746aa6584 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:53:05.019280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:53:05.065440 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee70348-4e29-45df-8ef0-951746aa6584 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:53:05.099955 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8efc2-197a-4f71-95c7-6ebd0994772c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:53:05.100545 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8efc2-197a-4f71-95c7-6ebd0994772c None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:05.100834 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8efc2-197a-4f71-95c7-6ebd0994772c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:05.101113 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8efc2-197a-4f71-95c7-6ebd0994772c None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:05.101909 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad8efc2-197a-4f71-95c7-6ebd0994772c None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:53:05.103293 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad8efc2-197a-4f71-95c7-6ebd0994772c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:53:05.119467 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 576/1169] 10.222.0.52 () {60 vars in 1138 bytes} [Thu Jul 21 15:53:04 2022] POST /identity/v3/domains => generated 267 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:53:07.236965 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c8be7-3d22-4ad4-87db-ccc341b2aa0d None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:07.236965 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c8be7-3d22-4ad4-87db-ccc341b2aa0d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:07.236965 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c8be7-3d22-4ad4-87db-ccc341b2aa0d None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:07.239015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 594/1170] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:53:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:53:07.244628 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff84d6-ef37-4c65-b141-000c29843a84 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:07.245101 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff84d6-ef37-4c65-b141-000c29843a84 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:07.245233 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff84d6-ef37-4c65-b141-000c29843a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:07.262891 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5dff84d6-ef37-4c65-b141-000c29843a84 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:53:07.264120 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5dff84d6-ef37-4c65-b141-000c29843a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:53:07.332085 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 577/1171] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:53:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 15:53:07.356424 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c8be7-3d22-4ad4-87db-ccc341b2aa0d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:53:07.378146 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:53:07.459462 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c8be7-3d22-4ad4-87db-ccc341b2aa0d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:53:07.494388 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ed0bb-0c06-4d48-bb9b-edb8e8a22791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:53:07.495208 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ed0bb-0c06-4d48-bb9b-edb8e8a22791 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:07.495493 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ed0bb-0c06-4d48-bb9b-edb8e8a22791 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:07.495769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ed0bb-0c06-4d48-bb9b-edb8e8a22791 None admin] PATH_INFO: `/v3/domains/bc5d66c0f0e84eaf8c6f887e09fb27ec` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:07.506230 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6ed0bb-0c06-4d48-bb9b-edb8e8a22791 None admin] RBAC: Authorizing `identity:get_domain(domain_id=bc5d66c0f0e84eaf8c6f887e09fb27ec)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:53:07.507753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6ed0bb-0c06-4d48-bb9b-edb8e8a22791 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:53:07.509410 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 595/1172] 10.222.0.52 () {56 vars in 1183 bytes} [Thu Jul 21 15:53:07 2022] GET /identity/v3/domains/bc5d66c0f0e84eaf8c6f887e09fb27ec => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:53:07.515352 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff84d6-ef37-4c65-b141-000c29843a84 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:53:07.560854 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:53:07.614639 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff84d6-ef37-4c65-b141-000c29843a84 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:53:07.647686 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ad3d3-24bc-41ba-bb39-cc9ab3a440f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:53:07.648344 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ad3d3-24bc-41ba-bb39-cc9ab3a440f4 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:07.648630 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ad3d3-24bc-41ba-bb39-cc9ab3a440f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:07.648910 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ad3d3-24bc-41ba-bb39-cc9ab3a440f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:07.649663 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ad3d3-24bc-41ba-bb39-cc9ab3a440f4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:53:07.651090 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ad3d3-24bc-41ba-bb39-cc9ab3a440f4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:53:07.680340 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 578/1173] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 21 15:53:07 2022] POST /identity/v3/projects => generated 361 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:53:09.763909 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5894016-774e-4850-a193-3917f5ec364a None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:09.764449 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5894016-774e-4850-a193-3917f5ec364a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:09.764449 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5894016-774e-4850-a193-3917f5ec364a None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:09.765832 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 596/1174] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:53:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:53:09.771184 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84340c87-4ea5-43b8-893e-2494929d077f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:09.771629 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84340c87-4ea5-43b8-893e-2494929d077f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:09.771752 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84340c87-4ea5-43b8-893e-2494929d077f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:09.788676 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-84340c87-4ea5-43b8-893e-2494929d077f None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:53:09.789906 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-84340c87-4ea5-43b8-893e-2494929d077f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:53:09.857020 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 579/1175] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:53:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 15:53:09.881477 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5894016-774e-4850-a193-3917f5ec364a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:53:09.971759 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:53:09.993403 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5894016-774e-4850-a193-3917f5ec364a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:53:10.029286 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8368cf9-d811-4fea-9907-159fb167bf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:53:10.030098 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8368cf9-d811-4fea-9907-159fb167bf3f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:10.030380 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8368cf9-d811-4fea-9907-159fb167bf3f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:10.030653 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8368cf9-d811-4fea-9907-159fb167bf3f None admin] PATH_INFO: `/v3/domains/bc5d66c0f0e84eaf8c6f887e09fb27ec` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:10.031852 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8368cf9-d811-4fea-9907-159fb167bf3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=bc5d66c0f0e84eaf8c6f887e09fb27ec)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:53:10.033294 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8368cf9-d811-4fea-9907-159fb167bf3f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:53:10.034936 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 597/1176] 10.222.0.52 () {56 vars in 1183 bytes} [Thu Jul 21 15:53:09 2022] GET /identity/v3/domains/bc5d66c0f0e84eaf8c6f887e09fb27ec => generated 267 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:53:10.043009 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84340c87-4ea5-43b8-893e-2494929d077f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:53:10.070030 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:53:10.148667 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84340c87-4ea5-43b8-893e-2494929d077f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:53:10.183531 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb5560-9d33-4aa8-b98a-bb876290a11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:53:10.184188 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb5560-9d33-4aa8-b98a-bb876290a11a None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:10.184468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb5560-9d33-4aa8-b98a-bb876290a11a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:10.184739 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb5560-9d33-4aa8-b98a-bb876290a11a None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:10.185449 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb5560-9d33-4aa8-b98a-bb876290a11a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:53:10.186902 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb5560-9d33-4aa8-b98a-bb876290a11a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:53:10.205997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 580/1177] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 15:53:10 2022] POST /identity/v3/users => generated 311 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:53:12.339894 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2cfe9-ced3-41d4-8594-9033911d6796 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:12.340461 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2cfe9-ced3-41d4-8594-9033911d6796 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:12.340461 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2cfe9-ced3-41d4-8594-9033911d6796 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:12.341833 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 598/1178] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:53:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:53:12.347400 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e70e5c-b772-45bc-a7ea-2147874e7435 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:12.347809 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e70e5c-b772-45bc-a7ea-2147874e7435 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:12.347968 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e70e5c-b772-45bc-a7ea-2147874e7435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:12.366023 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-57e70e5c-b772-45bc-a7ea-2147874e7435 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:53:12.367224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-57e70e5c-b772-45bc-a7ea-2147874e7435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:53:12.433778 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 581/1179] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:53:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 15:53:12.456682 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2cfe9-ced3-41d4-8594-9033911d6796 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:53:12.557035 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2cfe9-ced3-41d4-8594-9033911d6796 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:53:12.593515 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc4e8f7-e896-4477-8b67-6890004dc0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:53:12.594334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4e8f7-e896-4477-8b67-6890004dc0ef None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:12.594614 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4e8f7-e896-4477-8b67-6890004dc0ef None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:12.594883 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4e8f7-e896-4477-8b67-6890004dc0ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:12.605505 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc4e8f7-e896-4477-8b67-6890004dc0ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:53:12.613120 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-cfc4e8f7-e896-4477-8b67-6890004dc0ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:53:12.613120 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:53:12.613120 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:53:12.613120 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:53:12.613120 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:53:12.613120 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:53:12.613120 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:53:12.613120 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:53:12.613120 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:53:12.613120 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:53:12.613120 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:53:12.613120 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:53:12.613120 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:53:12.613120 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:53:12.613120 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:53:12.613120 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:53:12.614291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:53:12.614291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:53:12.614291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:53:12.614291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:53:12.614291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:53:12.614291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:53:12.614291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:53:12.614291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:53:12.614291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:53:12.614291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:53:12.614291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:53:12.614291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:53:12.614291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:53:12.614291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:53:12.614291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:53:12.614291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:53:12.615395 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:53:12.615395 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:53:12.615395 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:53:12.615395 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:53:12.615395 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:53:12.615395 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:53:12.615395 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:53:12.615395 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:53:12.615395 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:53:12.615395 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:53:12.615395 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:53:12.615395 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:53:12.615395 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 599/1180] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:53:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:53:12.620869 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e70e5c-b772-45bc-a7ea-2147874e7435 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:53:12.635039 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:53:12.720966 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e70e5c-b772-45bc-a7ea-2147874e7435 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:53:12.754338 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f55e8d-03de-44d4-8eac-64e940e514a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:53:12.754997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f55e8d-03de-44d4-8eac-64e940e514a9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:12.755427 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f55e8d-03de-44d4-8eac-64e940e514a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:12.755720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f55e8d-03de-44d4-8eac-64e940e514a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:12.756480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f55e8d-03de-44d4-8eac-64e940e514a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:53:12.756791 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f55e8d-03de-44d4-8eac-64e940e514a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:53:12.758225 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f55e8d-03de-44d4-8eac-64e940e514a9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:53:12.766663 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 582/1181] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:53:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:53:12.772445 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc4e8f7-e896-4477-8b67-6890004dc0ef None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:53:12.784375 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:53:12.875104 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc4e8f7-e896-4477-8b67-6890004dc0ef None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:53:12.907917 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc61c2-34c3-42cf-a4db-1704c8954c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:53:12.908790 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc61c2-34c3-42cf-a4db-1704c8954c38 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:12.909071 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc61c2-34c3-42cf-a4db-1704c8954c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:12.909341 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc61c2-34c3-42cf-a4db-1704c8954c38 None admin] PATH_INFO: `/v3/users/4e972e1f5a244a68a03fb433a0c008da` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:12.923673 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc61c2-34c3-42cf-a4db-1704c8954c38 None admin] RBAC: Authorizing `identity:get_user(user_id=4e972e1f5a244a68a03fb433a0c008da)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:53:12.925120 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc61c2-34c3-42cf-a4db-1704c8954c38 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:53:12.926854 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 600/1182] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:53:12 2022] GET /identity/v3/users/4e972e1f5a244a68a03fb433a0c008da => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:53:12.933360 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f55e8d-03de-44d4-8eac-64e940e514a9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:53:13.015371 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:53:13.039647 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f55e8d-03de-44d4-8eac-64e940e514a9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:53:13.072863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f5e57-0891-479c-a142-99d86d369c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:53:13.073537 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f5e57-0891-479c-a142-99d86d369c12 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:13.073852 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f5e57-0891-479c-a142-99d86d369c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:13.074133 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f5e57-0891-479c-a142-99d86d369c12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:13.140189 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f5e57-0891-479c-a142-99d86d369c12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:53:13.141559 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f5e57-0891-479c-a142-99d86d369c12 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:53:13.214954 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:53:13.245662 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 583/1183] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:53:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 15:53:13.251506 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc61c2-34c3-42cf-a4db-1704c8954c38 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:53:13.318219 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:53:13.354364 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc61c2-34c3-42cf-a4db-1704c8954c38 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:53:13.389245 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba728bf-fb06-4cad-9fce-77639a52937b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:53:13.390131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba728bf-fb06-4cad-9fce-77639a52937b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:13.390471 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba728bf-fb06-4cad-9fce-77639a52937b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:13.390698 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba728bf-fb06-4cad-9fce-77639a52937b None admin] PATH_INFO: `/v3/projects/62b74c28cc3f4fa9ad0b354a3d0f4701` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:13.391887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba728bf-fb06-4cad-9fce-77639a52937b None admin] RBAC: Authorizing `identity:get_project(project_id=62b74c28cc3f4fa9ad0b354a3d0f4701)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:53:13.393338 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba728bf-fb06-4cad-9fce-77639a52937b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:53:13.395071 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 601/1184] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:53:13 2022] GET /identity/v3/projects/62b74c28cc3f4fa9ad0b354a3d0f4701 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:53:13.401287 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f5e57-0891-479c-a142-99d86d369c12 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:53:13.509141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f5e57-0891-479c-a142-99d86d369c12 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:53:13.542414 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1afd98-812e-44d2-8ee2-89d24fd963f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:53:13.543055 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1afd98-812e-44d2-8ee2-89d24fd963f5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:13.543340 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1afd98-812e-44d2-8ee2-89d24fd963f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:13.543614 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1afd98-812e-44d2-8ee2-89d24fd963f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:13.544515 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1afd98-812e-44d2-8ee2-89d24fd963f5 None admin] RBAC: Adding query filter params (role.id=475521b8a0174037b162f92022aea1a1, scope.project.id=62b74c28cc3f4fa9ad0b354a3d0f4701, user.id=4e972e1f5a244a68a03fb433a0c008da) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:53:13.544827 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1afd98-812e-44d2-8ee2-89d24fd963f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:53:13.546279 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1afd98-812e-44d2-8ee2-89d24fd963f5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:53:13.551535 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 584/1185] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:53:13 2022] GET /identity/v3/role_assignments?user.id=4e972e1f5a244a68a03fb433a0c008da&scope.project.id=62b74c28cc3f4fa9ad0b354a3d0f4701&role.id=475521b8a0174037b162f92022aea1a1 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:53:15.555265 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c382c02-c17b-43ff-abcb-8b29db4bc79e None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:15.555771 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c382c02-c17b-43ff-abcb-8b29db4bc79e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:15.555771 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c382c02-c17b-43ff-abcb-8b29db4bc79e None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:15.557097 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 602/1186] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:53:15.562404 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46b8c5-bd02-4e97-8e82-a103225a9db1 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:15.562840 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46b8c5-bd02-4e97-8e82-a103225a9db1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:15.562955 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46b8c5-bd02-4e97-8e82-a103225a9db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:15.579661 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-fc46b8c5-bd02-4e97-8e82-a103225a9db1 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:53:15.580865 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fc46b8c5-bd02-4e97-8e82-a103225a9db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:53:15.612412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:53:15.649709 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 585/1187] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:53:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:53:15.674116 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c382c02-c17b-43ff-abcb-8b29db4bc79e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:53:15.778141 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c382c02-c17b-43ff-abcb-8b29db4bc79e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:53:15.820633 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a042ae-3452-4b25-803f-66698097056e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:53:15.821332 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a042ae-3452-4b25-803f-66698097056e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:15.821637 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a042ae-3452-4b25-803f-66698097056e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:15.821915 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a042ae-3452-4b25-803f-66698097056e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:15.824166 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:53:15.833369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a042ae-3452-4b25-803f-66698097056e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:53:15.841919 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-94a042ae-3452-4b25-803f-66698097056e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:53:15.841919 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:53:15.841919 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:53:15.841919 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:53:15.841919 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:53:15.841919 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:53:15.841919 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:53:15.841919 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:53:15.841919 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:53:15.841919 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:53:15.841919 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:53:15.841919 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:53:15.841919 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:53:15.841919 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:53:15.841919 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:53:15.841919 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:53:15.843096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:53:15.843096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:53:15.843096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:53:15.843096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:53:15.843096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:53:15.843096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:53:15.843096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:53:15.843096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:53:15.843096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:53:15.843096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:53:15.843096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:53:15.843096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:53:15.843096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:53:15.843096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:53:15.843096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:53:15.843096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:53:15.844034 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:53:15.844034 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:53:15.844034 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:53:15.844034 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:53:15.844034 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:53:15.844034 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:53:15.844034 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:53:15.844034 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:53:15.844034 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:53:15.844034 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:53:15.844034 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:53:15.844034 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:53:15.844034 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 603/1188] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:53:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:53:15.849633 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc46b8c5-bd02-4e97-8e82-a103225a9db1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:53:15.950172 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc46b8c5-bd02-4e97-8e82-a103225a9db1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:53:15.984035 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a663b4-8658-420c-940f-0b224cf4ca39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:53:15.984703 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a663b4-8658-420c-940f-0b224cf4ca39 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:15.984988 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a663b4-8658-420c-940f-0b224cf4ca39 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:15.985267 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a663b4-8658-420c-940f-0b224cf4ca39 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:15.986054 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a663b4-8658-420c-940f-0b224cf4ca39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:53:15.986369 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a663b4-8658-420c-940f-0b224cf4ca39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:53:15.987758 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a663b4-8658-420c-940f-0b224cf4ca39 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:53:15.996035 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 586/1189] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:53:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:53:16.001957 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a042ae-3452-4b25-803f-66698097056e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:53:16.098985 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a042ae-3452-4b25-803f-66698097056e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:53:16.130887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f23c5-1d1a-42d2-9647-23eb59adebaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:53:16.131777 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f23c5-1d1a-42d2-9647-23eb59adebaa None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:16.132058 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f23c5-1d1a-42d2-9647-23eb59adebaa None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:16.132331 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f23c5-1d1a-42d2-9647-23eb59adebaa None admin] PATH_INFO: `/v3/users/4e972e1f5a244a68a03fb433a0c008da` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:16.133635 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1f23c5-1d1a-42d2-9647-23eb59adebaa None admin] RBAC: Authorizing `identity:get_user(user_id=4e972e1f5a244a68a03fb433a0c008da)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:53:16.135118 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1f23c5-1d1a-42d2-9647-23eb59adebaa None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:53:16.137341 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 604/1190] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:53:16 2022] GET /identity/v3/users/4e972e1f5a244a68a03fb433a0c008da => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:53:16.143327 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a663b4-8658-420c-940f-0b224cf4ca39 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:53:16.174335 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:53:16.243174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a663b4-8658-420c-940f-0b224cf4ca39 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:53:16.276966 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05889a50-89e6-4643-bb7b-2115bd98c9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:53:16.277848 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05889a50-89e6-4643-bb7b-2115bd98c9be None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:16.278133 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05889a50-89e6-4643-bb7b-2115bd98c9be None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:16.278410 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05889a50-89e6-4643-bb7b-2115bd98c9be None admin] PATH_INFO: `/v3/projects/62b74c28cc3f4fa9ad0b354a3d0f4701` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:16.279608 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05889a50-89e6-4643-bb7b-2115bd98c9be None admin] RBAC: Authorizing `identity:get_project(project_id=62b74c28cc3f4fa9ad0b354a3d0f4701)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:53:16.281058 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05889a50-89e6-4643-bb7b-2115bd98c9be None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:53:16.282805 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 587/1191] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:53:16 2022] GET /identity/v3/projects/62b74c28cc3f4fa9ad0b354a3d0f4701 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:53:16.288820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f23c5-1d1a-42d2-9647-23eb59adebaa None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:53:16.360440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:53:16.406049 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f23c5-1d1a-42d2-9647-23eb59adebaa None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:53:16.439651 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888cf46-969a-4029-ae78-50090c632f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:53:16.440541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888cf46-969a-4029-ae78-50090c632f98 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:16.440821 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888cf46-969a-4029-ae78-50090c632f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:16.441098 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888cf46-969a-4029-ae78-50090c632f98 None admin] PATH_INFO: `/v3/projects/62b74c28cc3f4fa9ad0b354a3d0f4701/users/4e972e1f5a244a68a03fb433a0c008da/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:16.443291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3888cf46-969a-4029-ae78-50090c632f98 None admin] RBAC: Authorizing `identity:create_grant(project_id=62b74c28cc3f4fa9ad0b354a3d0f4701, user_id=4e972e1f5a244a68a03fb433a0c008da, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:53:16.444896 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3888cf46-969a-4029-ae78-50090c632f98 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:53:16.459720 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 605/1192] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 15:53:16 2022] PUT /identity/v3/projects/62b74c28cc3f4fa9ad0b354a3d0f4701/users/4e972e1f5a244a68a03fb433a0c008da/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 15:53:18.471468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89619565-da3c-4ad2-820a-548293a61e7d None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:18.471997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89619565-da3c-4ad2-820a-548293a61e7d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:18.471997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89619565-da3c-4ad2-820a-548293a61e7d None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:18.473390 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 588/1193] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:53:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:53:18.478575 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f938c35-7606-42ae-a160-430047f252d3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:18.478962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f938c35-7606-42ae-a160-430047f252d3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:18.479682 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f938c35-7606-42ae-a160-430047f252d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:18.496771 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1f938c35-7606-42ae-a160-430047f252d3 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:53:18.497926 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1f938c35-7606-42ae-a160-430047f252d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:53:18.564935 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 606/1194] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:53:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:53:18.593829 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89619565-da3c-4ad2-820a-548293a61e7d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:53:18.689015 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:53:18.694247 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89619565-da3c-4ad2-820a-548293a61e7d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:53:18.728306 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb235264-1f5e-4dae-89eb-65e802013d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:53:18.729705 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb235264-1f5e-4dae-89eb-65e802013d6d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:18.729996 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb235264-1f5e-4dae-89eb-65e802013d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:18.730274 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb235264-1f5e-4dae-89eb-65e802013d6d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:18.740165 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb235264-1f5e-4dae-89eb-65e802013d6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:53:18.747869 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-eb235264-1f5e-4dae-89eb-65e802013d6d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:53:18.747869 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:53:18.747869 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:53:18.747869 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:53:18.747869 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:53:18.747869 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:53:18.747869 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:53:18.747869 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:53:18.747869 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:53:18.747869 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:53:18.747869 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:53:18.747869 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:53:18.747869 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:53:18.747869 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:53:18.747869 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:53:18.747869 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:53:18.748997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:53:18.748997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:53:18.748997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:53:18.748997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:53:18.748997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:53:18.748997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:53:18.748997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:53:18.748997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:53:18.748997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:53:18.748997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:53:18.748997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:53:18.748997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:53:18.748997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:53:18.748997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:53:18.748997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:53:18.748997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:53:18.750143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:53:18.750143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:53:18.750143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:53:18.750143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:53:18.750143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:53:18.750143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:53:18.750143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:53:18.750143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:53:18.750143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:53:18.750143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:53:18.750143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:53:18.750143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:53:18.750143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 589/1195] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:53:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:53:18.755201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f938c35-7606-42ae-a160-430047f252d3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:53:18.806748 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:53:18.853670 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f938c35-7606-42ae-a160-430047f252d3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:53:18.892192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804fb93-9a44-495a-9e61-c4f48a5ad62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:53:18.892903 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804fb93-9a44-495a-9e61-c4f48a5ad62d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:18.893224 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804fb93-9a44-495a-9e61-c4f48a5ad62d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:18.893527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804fb93-9a44-495a-9e61-c4f48a5ad62d None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:18.894360 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5804fb93-9a44-495a-9e61-c4f48a5ad62d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:53:18.894709 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5804fb93-9a44-495a-9e61-c4f48a5ad62d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:53:18.896105 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5804fb93-9a44-495a-9e61-c4f48a5ad62d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:53:18.904988 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 607/1196] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:53:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:53:18.911005 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb235264-1f5e-4dae-89eb-65e802013d6d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:53:19.011341 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb235264-1f5e-4dae-89eb-65e802013d6d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:53:19.044852 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c837c-79f8-4326-90ca-920b8b96f114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:53:19.045774 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c837c-79f8-4326-90ca-920b8b96f114 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:19.046058 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c837c-79f8-4326-90ca-920b8b96f114 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:19.046340 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c837c-79f8-4326-90ca-920b8b96f114 None admin] PATH_INFO: `/v3/users/4e972e1f5a244a68a03fb433a0c008da` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:19.047565 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c837c-79f8-4326-90ca-920b8b96f114 None admin] RBAC: Authorizing `identity:get_user(user_id=4e972e1f5a244a68a03fb433a0c008da)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:53:19.049015 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c837c-79f8-4326-90ca-920b8b96f114 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:53:19.050727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 590/1197] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 15:53:18 2022] GET /identity/v3/users/4e972e1f5a244a68a03fb433a0c008da => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:53:19.056765 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804fb93-9a44-495a-9e61-c4f48a5ad62d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:53:19.159111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804fb93-9a44-495a-9e61-c4f48a5ad62d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:53:19.177017 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:53:19.198288 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b610596-ca73-48ed-9e24-f498a4327a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:53:19.198937 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b610596-ca73-48ed-9e24-f498a4327a78 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:19.199225 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b610596-ca73-48ed-9e24-f498a4327a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:19.199503 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b610596-ca73-48ed-9e24-f498a4327a78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:19.276556 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b610596-ca73-48ed-9e24-f498a4327a78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:53:19.277984 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b610596-ca73-48ed-9e24-f498a4327a78 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:53:19.373246 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 608/1198] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:53:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 15:53:19.379192 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c837c-79f8-4326-90ca-920b8b96f114 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:53:19.407432 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:53:19.478489 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c837c-79f8-4326-90ca-920b8b96f114 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:53:19.511107 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e84d3-2e71-487a-95f7-d4b528903986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:53:19.511958 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e84d3-2e71-487a-95f7-d4b528903986 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:19.512240 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e84d3-2e71-487a-95f7-d4b528903986 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:19.512516 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e84d3-2e71-487a-95f7-d4b528903986 None admin] PATH_INFO: `/v3/projects/62b74c28cc3f4fa9ad0b354a3d0f4701` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:19.514225 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e84d3-2e71-487a-95f7-d4b528903986 None admin] RBAC: Authorizing `identity:get_project(project_id=62b74c28cc3f4fa9ad0b354a3d0f4701)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:53:19.515644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e84d3-2e71-487a-95f7-d4b528903986 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:53:19.517334 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 591/1199] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 15:53:19 2022] GET /identity/v3/projects/62b74c28cc3f4fa9ad0b354a3d0f4701 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:53:19.523504 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b610596-ca73-48ed-9e24-f498a4327a78 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:53:19.543636 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:53:19.634109 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b610596-ca73-48ed-9e24-f498a4327a78 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:53:19.666999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e617194-70a0-4402-9062-cafca32e0f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:53:19.667654 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e617194-70a0-4402-9062-cafca32e0f0f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:53:19.667938 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e617194-70a0-4402-9062-cafca32e0f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:53:19.668214 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e617194-70a0-4402-9062-cafca32e0f0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:53:19.669054 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e617194-70a0-4402-9062-cafca32e0f0f None admin] RBAC: Adding query filter params (role.id=475521b8a0174037b162f92022aea1a1, scope.project.id=62b74c28cc3f4fa9ad0b354a3d0f4701, user.id=4e972e1f5a244a68a03fb433a0c008da) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:53:19.669378 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e617194-70a0-4402-9062-cafca32e0f0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:53:19.670866 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e617194-70a0-4402-9062-cafca32e0f0f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:53:19.677155 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 609/1200] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:53:19 2022] GET /identity/v3/role_assignments?user.id=4e972e1f5a244a68a03fb433a0c008da&scope.project.id=62b74c28cc3f4fa9ad0b354a3d0f4701&role.id=475521b8a0174037b162f92022aea1a1 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:53:29.507021 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=53 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:53:29.674347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=61 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:16.933884 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca237e9c-c8b3-4ee8-b3e0-744b25093500 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:16.934506 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca237e9c-c8b3-4ee8-b3e0-744b25093500 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:16.934780 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca237e9c-c8b3-4ee8-b3e0-744b25093500 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:16.936882 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 592/1201] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:54:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:54:16.943361 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d02f5-b37d-478c-b464-3ca3075d414e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:16.943908 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d02f5-b37d-478c-b464-3ca3075d414e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:16.944113 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d02f5-b37d-478c-b464-3ca3075d414e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:16.967962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-078d02f5-b37d-478c-b464-3ca3075d414e None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:54:16.969356 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-078d02f5-b37d-478c-b464-3ca3075d414e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:54:17.061396 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 610/1202] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:54:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 15:54:17.089757 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca237e9c-c8b3-4ee8-b3e0-744b25093500 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:17.249891 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca237e9c-c8b3-4ee8-b3e0-744b25093500 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:17.301561 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba001e79-359a-48a5-a22f-fb9b819e4a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:17.302801 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba001e79-359a-48a5-a22f-fb9b819e4a3a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:17.303287 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba001e79-359a-48a5-a22f-fb9b819e4a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:17.303768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba001e79-359a-48a5-a22f-fb9b819e4a3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:17.321867 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba001e79-359a-48a5-a22f-fb9b819e4a3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:17.323532 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba001e79-359a-48a5-a22f-fb9b819e4a3a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:17.344602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ba001e79-359a-48a5-a22f-fb9b819e4a3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:54:17.344602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:17.344602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:17.344602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:17.344602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:17.344602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:17.344602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:17.344602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:17.344602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:17.344602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:17.344602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:17.344602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:17.344602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:54:17.344602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:54:17.344602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:54:17.346138 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:54:17.346138 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:17.346138 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:17.346138 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:17.346138 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:17.346138 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:17.346138 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:17.346138 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:17.346138 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:17.346138 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:17.346138 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:17.346138 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:17.346138 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:17.346138 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:17.346138 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:17.346138 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:17.346138 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:17.356018 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:54:17.356018 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:54:17.356018 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:54:17.356018 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:54:17.356018 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 593/1203] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:54:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:54:17.356402 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d02f5-b37d-478c-b464-3ca3075d414e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:17.511820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d02f5-b37d-478c-b464-3ca3075d414e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:17.550422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:17.566056 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35330caa-0bda-432d-bffd-14c64e54e1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:17.566809 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35330caa-0bda-432d-bffd-14c64e54e1c4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:17.567095 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35330caa-0bda-432d-bffd-14c64e54e1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:17.567442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35330caa-0bda-432d-bffd-14c64e54e1c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:17.568263 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35330caa-0bda-432d-bffd-14c64e54e1c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:17.568541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35330caa-0bda-432d-bffd-14c64e54e1c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:17.570188 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35330caa-0bda-432d-bffd-14c64e54e1c4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:17.584993 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 611/1204] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:17.591364 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba001e79-359a-48a5-a22f-fb9b819e4a3a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:17.667491 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:17.740242 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba001e79-359a-48a5-a22f-fb9b819e4a3a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:17.784158 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42b97d-878d-4b39-bcf2-40c257fa91e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:17.789631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42b97d-878d-4b39-bcf2-40c257fa91e0 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:17.789631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42b97d-878d-4b39-bcf2-40c257fa91e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:17.789631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42b97d-878d-4b39-bcf2-40c257fa91e0 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:17.789631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42b97d-878d-4b39-bcf2-40c257fa91e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:17.789631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42b97d-878d-4b39-bcf2-40c257fa91e0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:17.820633 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 594/1205] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 15:54:17 2022] POST /identity/v3/users => generated 251 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:54:20.115575 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca72985-1cde-4625-b532-facce8165bf2 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:20.116316 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca72985-1cde-4625-b532-facce8165bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:20.116316 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca72985-1cde-4625-b532-facce8165bf2 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:20.117683 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 612/1206] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:54:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:54:20.124318 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd0ca3-3243-4e6b-8d75-67934fbf8168 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:20.124898 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd0ca3-3243-4e6b-8d75-67934fbf8168 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:20.124898 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd0ca3-3243-4e6b-8d75-67934fbf8168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:20.153947 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f9fd0ca3-3243-4e6b-8d75-67934fbf8168 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:54:20.155202 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f9fd0ca3-3243-4e6b-8d75-67934fbf8168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:54:20.260322 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 595/1207] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:54:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:54:20.293559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca72985-1cde-4625-b532-facce8165bf2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:20.438865 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca72985-1cde-4625-b532-facce8165bf2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:20.484643 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c8757-f0e9-4c21-a53f-3bbfda0c44ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:20.485497 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c8757-f0e9-4c21-a53f-3bbfda0c44ff None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:20.485824 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c8757-f0e9-4c21-a53f-3bbfda0c44ff None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:20.486083 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c8757-f0e9-4c21-a53f-3bbfda0c44ff None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:20.506445 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12c8757-f0e9-4c21-a53f-3bbfda0c44ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:54:20.517347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d12c8757-f0e9-4c21-a53f-3bbfda0c44ff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:54:20.517347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:20.517347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:20.517347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:20.517347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:20.517347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:20.517347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:20.517347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:20.517347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:20.517347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:20.517347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:20.517347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:20.517347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:54:20.517347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:54:20.517347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:54:20.517347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:54:20.519425 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:54:20.519425 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:54:20.519425 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:54:20.519425 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:54:20.519425 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:20.519425 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:20.519425 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:20.519425 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:20.519425 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:20.519425 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:20.519425 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:20.519425 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:20.519425 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:20.519425 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:20.519425 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:20.519425 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:20.521135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:20.521135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:20.521135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:20.521135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:20.521135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:54:20.521135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:54:20.521135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:54:20.521135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:54:20.521135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:54:20.521135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:54:20.521135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:54:20.521135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:54:20.524819 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd0ca3-3243-4e6b-8d75-67934fbf8168 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:20.525643 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 613/1208] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 15:54:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:54:20.578897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:20.664694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd0ca3-3243-4e6b-8d75-67934fbf8168 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:20.707765 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d0b62-6fb6-4acf-87fa-9d9c62b54a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:20.708478 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d0b62-6fb6-4acf-87fa-9d9c62b54a24 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:20.708782 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d0b62-6fb6-4acf-87fa-9d9c62b54a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:20.709149 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d0b62-6fb6-4acf-87fa-9d9c62b54a24 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:20.710151 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d0b62-6fb6-4acf-87fa-9d9c62b54a24 None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:20.710484 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d0b62-6fb6-4acf-87fa-9d9c62b54a24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:20.711978 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d0b62-6fb6-4acf-87fa-9d9c62b54a24 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:20.722635 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 596/1209] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 15:54:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:54:20.729542 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c8757-f0e9-4c21-a53f-3bbfda0c44ff None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:20.798855 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:20.880663 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c8757-f0e9-4c21-a53f-3bbfda0c44ff None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:20.926800 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee4de0a-de6c-444b-b2ff-78880a352de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:20.926800 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4de0a-de6c-444b-b2ff-78880a352de1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:20.926800 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4de0a-de6c-444b-b2ff-78880a352de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:20.926800 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4de0a-de6c-444b-b2ff-78880a352de1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:20.944079 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee4de0a-de6c-444b-b2ff-78880a352de1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:20.945816 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee4de0a-de6c-444b-b2ff-78880a352de1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:20.966131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-dee4de0a-de6c-444b-b2ff-78880a352de1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:54:20.966131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:20.966131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:20.966131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:20.966131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:20.966131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:20.966131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:20.966131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:20.966131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:20.966131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:20.966131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:20.966131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:20.966131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:54:20.966131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:54:20.966131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:54:20.967311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:54:20.967311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:20.967311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:20.967311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:20.967311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:20.967311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:20.967311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:20.967311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:20.967311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:20.967311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:20.967311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:20.967311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:20.967311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:20.967311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:20.967311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:20.967311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:20.967311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:20.968476 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:54:20.968476 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:54:20.968476 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:54:20.968476 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:54:20.969770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 614/1210] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:54:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:54:20.974564 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d0b62-6fb6-4acf-87fa-9d9c62b54a24 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:21.094194 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:21.097631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d0b62-6fb6-4acf-87fa-9d9c62b54a24 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:21.136698 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8699b395-3da9-4ac5-a661-105032daa355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:21.137512 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699b395-3da9-4ac5-a661-105032daa355 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:21.137880 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699b395-3da9-4ac5-a661-105032daa355 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:21.138135 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699b395-3da9-4ac5-a661-105032daa355 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:21.139145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8699b395-3da9-4ac5-a661-105032daa355 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:21.139469 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8699b395-3da9-4ac5-a661-105032daa355 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:21.140860 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8699b395-3da9-4ac5-a661-105032daa355 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:21.159353 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 597/1211] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:54:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:21.161157 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee4de0a-de6c-444b-b2ff-78880a352de1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:21.301560 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee4de0a-de6c-444b-b2ff-78880a352de1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:21.303540 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:21.357182 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd1ead-23b1-4b32-8031-81ed93dfba9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:21.358062 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd1ead-23b1-4b32-8031-81ed93dfba9c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:21.358344 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd1ead-23b1-4b32-8031-81ed93dfba9c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:21.358619 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd1ead-23b1-4b32-8031-81ed93dfba9c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:21.367355 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd1ead-23b1-4b32-8031-81ed93dfba9c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:21.368939 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd1ead-23b1-4b32-8031-81ed93dfba9c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:21.385709 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b7dd1ead-23b1-4b32-8031-81ed93dfba9c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 15:54:21.385709 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:21.385709 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:21.385709 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:21.385709 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:21.385709 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:21.385709 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:21.385709 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:21.385709 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:21.385709 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:21.385709 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:21.385709 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:21.385709 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:54:21.385709 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:54:21.385709 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:54:21.385709 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:54:21.387492 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:21.387492 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:21.387492 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:54:21.387492 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:54:21.387492 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:54:21.387492 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:54:21.387492 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:21.387492 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:21.387492 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:21.387492 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:21.387492 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:21.387492 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:21.387492 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:21.387492 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:21.387492 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:21.387492 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:21.387492 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:21.387492 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:21.389335 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:21.389335 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:21.389335 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:54:21.389335 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:54:21.389335 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:54:21.389335 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:54:21.389335 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:54:21.389335 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:54:21.389335 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 15:54:21.389335 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:54:21.389335 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 615/1212] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:54:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:54:21.393484 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8699b395-3da9-4ac5-a661-105032daa355 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:21.522615 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8699b395-3da9-4ac5-a661-105032daa355 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:21.571090 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb8dd5b-5e26-4377-836e-2b81276f9b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:21.573657 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb8dd5b-5e26-4377-836e-2b81276f9b4f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:21.573657 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb8dd5b-5e26-4377-836e-2b81276f9b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:21.573657 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb8dd5b-5e26-4377-836e-2b81276f9b4f None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:21.575342 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb8dd5b-5e26-4377-836e-2b81276f9b4f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:21.575342 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb8dd5b-5e26-4377-836e-2b81276f9b4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:21.576748 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb8dd5b-5e26-4377-836e-2b81276f9b4f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:21.608125 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 598/1213] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Jul 21 15:54:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:54:21.614927 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd1ead-23b1-4b32-8031-81ed93dfba9c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:21.750576 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd1ead-23b1-4b32-8031-81ed93dfba9c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:21.791442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aede3aa-d243-4e2a-bf91-338911e48543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:21.791442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aede3aa-d243-4e2a-bf91-338911e48543 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:21.791442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aede3aa-d243-4e2a-bf91-338911e48543 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:21.791806 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aede3aa-d243-4e2a-bf91-338911e48543 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:21.824659 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:21.870818 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aede3aa-d243-4e2a-bf91-338911e48543 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:21.872683 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aede3aa-d243-4e2a-bf91-338911e48543 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:22.027820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 616/1214] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:54:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 15:54:22.034337 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb8dd5b-5e26-4377-836e-2b81276f9b4f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:22.063826 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:22.190513 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb8dd5b-5e26-4377-836e-2b81276f9b4f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:22.242393 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa946c-ac26-4e47-8575-de5afbf9466f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:22.243332 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa946c-ac26-4e47-8575-de5afbf9466f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:22.243605 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa946c-ac26-4e47-8575-de5afbf9466f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:22.243963 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa946c-ac26-4e47-8575-de5afbf9466f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:22.262323 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fa946c-ac26-4e47-8575-de5afbf9466f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:22.264126 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fa946c-ac26-4e47-8575-de5afbf9466f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:22.299759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-a9fa946c-ac26-4e47-8575-de5afbf9466f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:54:22.299759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:22.299759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:22.299759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:22.299759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:22.299759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:22.299759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:22.299759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:22.299759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:22.299759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:22.299759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:22.299759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:22.299759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:54:22.299759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:54:22.299759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:54:22.300815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:54:22.300815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:22.300815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:22.300815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:22.300815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:22.300815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:22.300815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:22.300815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:22.300815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:22.300815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:22.300815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:22.300815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:22.300815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:22.300815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:22.300815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:22.300815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:22.300815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:22.301793 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:54:22.301793 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:54:22.301793 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:54:22.301793 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:54:22.306763 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 599/1215] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:54:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:54:22.308108 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aede3aa-d243-4e2a-bf91-338911e48543 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:22.398204 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:22.449680 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aede3aa-d243-4e2a-bf91-338911e48543 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:22.493366 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f7532-2a9c-46ed-a41c-5ea4d34a8714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:22.494264 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f7532-2a9c-46ed-a41c-5ea4d34a8714 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:22.496173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f7532-2a9c-46ed-a41c-5ea4d34a8714 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:22.496173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f7532-2a9c-46ed-a41c-5ea4d34a8714 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:22.496173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f7532-2a9c-46ed-a41c-5ea4d34a8714 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:22.496173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f7532-2a9c-46ed-a41c-5ea4d34a8714 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:22.497497 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f7532-2a9c-46ed-a41c-5ea4d34a8714 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:22.512189 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 617/1216] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:22.517981 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa946c-ac26-4e47-8575-de5afbf9466f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:22.608612 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:22.643588 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa946c-ac26-4e47-8575-de5afbf9466f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:22.694388 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc9567-69c0-464d-97a0-c0329941ad7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:22.695241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc9567-69c0-464d-97a0-c0329941ad7e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:22.695496 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc9567-69c0-464d-97a0-c0329941ad7e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:22.695767 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc9567-69c0-464d-97a0-c0329941ad7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:22.703386 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fc9567-69c0-464d-97a0-c0329941ad7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:22.704807 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fc9567-69c0-464d-97a0-c0329941ad7e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:22.719330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-d0fc9567-69c0-464d-97a0-c0329941ad7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:54:22.719330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:22.719330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:22.719330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:22.719330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:22.719330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:22.719330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:22.719330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:22.719330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:22.719330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:22.719330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:22.719330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:22.719330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:54:22.719330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:54:22.719330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:54:22.721208 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:54:22.721208 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:22.721208 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:22.721208 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:22.721208 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:22.721208 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:22.721208 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:22.721208 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:22.721208 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:22.721208 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:22.721208 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:22.721208 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:22.721208 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:22.721208 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:22.721208 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:22.721208 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:22.721208 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:22.722483 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:54:22.722483 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:54:22.722483 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:54:22.722483 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:54:22.722483 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:54:22.722483 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:54:22.722483 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:54:22.722483 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:54:22.722483 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 600/1217] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:54:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:54:22.729040 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f7532-2a9c-46ed-a41c-5ea4d34a8714 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:22.857713 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f7532-2a9c-46ed-a41c-5ea4d34a8714 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:22.902728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:22.917329 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb9ebd-5515-41c0-9fb2-a6d93049907d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:22.918135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb9ebd-5515-41c0-9fb2-a6d93049907d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:22.918487 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb9ebd-5515-41c0-9fb2-a6d93049907d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:22.918830 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb9ebd-5515-41c0-9fb2-a6d93049907d None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:22.920495 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb9ebd-5515-41c0-9fb2-a6d93049907d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:22.920495 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb9ebd-5515-41c0-9fb2-a6d93049907d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:22.921798 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb9ebd-5515-41c0-9fb2-a6d93049907d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:22.942578 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 618/1218] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:54:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:22.949059 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc9567-69c0-464d-97a0-c0329941ad7e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:23.061867 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc9567-69c0-464d-97a0-c0329941ad7e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:23.093222 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:23.099942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7530a9f8-d43b-4022-934d-35db5d2256fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:23.100605 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530a9f8-d43b-4022-934d-35db5d2256fb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:23.100889 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530a9f8-d43b-4022-934d-35db5d2256fb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:23.101794 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530a9f8-d43b-4022-934d-35db5d2256fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:23.102647 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7530a9f8-d43b-4022-934d-35db5d2256fb None admin] RBAC: Adding query filter params (role.id=64d94d0d5a7640818c21aa5a622624ab, scope.project.id=bb1ac00308714c39a79aae09fbb73021, user.id=822d9d3521db4a9688f51c6313c7dd74) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:23.102963 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7530a9f8-d43b-4022-934d-35db5d2256fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:23.104426 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7530a9f8-d43b-4022-934d-35db5d2256fb None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:23.110793 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 601/1219] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:54:22 2022] GET /identity/v3/role_assignments?user.id=822d9d3521db4a9688f51c6313c7dd74&scope.project.id=bb1ac00308714c39a79aae09fbb73021&role.id=64d94d0d5a7640818c21aa5a622624ab => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:25.296822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33142f03-3e34-4811-a811-09e026a6d50e None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:25.297382 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33142f03-3e34-4811-a811-09e026a6d50e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:25.297382 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33142f03-3e34-4811-a811-09e026a6d50e None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:25.298814 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 619/1220] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:54:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 15:54:25.305067 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed4da7-66e4-44b6-a5ae-01f0c5233976 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:25.305595 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed4da7-66e4-44b6-a5ae-01f0c5233976 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:25.305712 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed4da7-66e4-44b6-a5ae-01f0c5233976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:25.325441 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-20ed4da7-66e4-44b6-a5ae-01f0c5233976 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:54:25.327007 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-20ed4da7-66e4-44b6-a5ae-01f0c5233976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:54:25.417378 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 602/1221] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:54:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:54:25.440596 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33142f03-3e34-4811-a811-09e026a6d50e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:25.558985 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33142f03-3e34-4811-a811-09e026a6d50e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:25.604907 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2f4b5-5587-4ea1-a717-0d547a3ead31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:25.605511 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2f4b5-5587-4ea1-a717-0d547a3ead31 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:25.605829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2f4b5-5587-4ea1-a717-0d547a3ead31 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:25.606108 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2f4b5-5587-4ea1-a717-0d547a3ead31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:25.619292 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2f4b5-5587-4ea1-a717-0d547a3ead31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:54:25.630654 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-1bf2f4b5-5587-4ea1-a717-0d547a3ead31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:54:25.630654 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:25.630654 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:25.630654 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:25.630654 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:25.630654 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:25.630654 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:25.630654 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:25.630654 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:25.630654 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:25.630654 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:25.630654 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:25.630654 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:54:25.630654 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:54:25.630654 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:54:25.630654 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:54:25.631822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:54:25.631822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:54:25.631822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:54:25.631822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:54:25.631822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:25.631822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:25.631822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:25.631822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:25.631822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:25.631822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:25.631822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:25.631822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:25.631822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:25.631822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:25.631822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:25.631822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:25.636451 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:25.636451 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:25.636451 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:25.636451 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:25.636451 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:54:25.636451 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:54:25.636451 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:54:25.636451 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:54:25.636451 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:54:25.636451 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:54:25.636451 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:54:25.636451 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:54:25.636451 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 620/1222] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 15:54:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:54:25.640184 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed4da7-66e4-44b6-a5ae-01f0c5233976 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:25.765688 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed4da7-66e4-44b6-a5ae-01f0c5233976 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:25.796883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:25.809688 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8929ae26-75a5-4a7c-b45d-a4ce14ddad3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:25.810364 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929ae26-75a5-4a7c-b45d-a4ce14ddad3a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:25.810713 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929ae26-75a5-4a7c-b45d-a4ce14ddad3a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:25.810988 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929ae26-75a5-4a7c-b45d-a4ce14ddad3a None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:25.811836 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8929ae26-75a5-4a7c-b45d-a4ce14ddad3a None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:25.812209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8929ae26-75a5-4a7c-b45d-a4ce14ddad3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:25.813700 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8929ae26-75a5-4a7c-b45d-a4ce14ddad3a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:25.823551 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 603/1223] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 15:54:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:25.830519 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2f4b5-5587-4ea1-a717-0d547a3ead31 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:25.902264 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:25.971243 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2f4b5-5587-4ea1-a717-0d547a3ead31 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:26.011575 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109b37a5-85ea-4d78-b719-fe9c2a46d22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:26.012451 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b37a5-85ea-4d78-b719-fe9c2a46d22d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:26.012746 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b37a5-85ea-4d78-b719-fe9c2a46d22d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:26.013101 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b37a5-85ea-4d78-b719-fe9c2a46d22d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:26.024606 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109b37a5-85ea-4d78-b719-fe9c2a46d22d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:26.026397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109b37a5-85ea-4d78-b719-fe9c2a46d22d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:26.048817 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-109b37a5-85ea-4d78-b719-fe9c2a46d22d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:54:26.048817 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:26.048817 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:26.048817 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:26.048817 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:26.048817 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:26.048817 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:26.048817 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:26.048817 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:26.048817 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:26.048817 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:26.048817 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:26.048817 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:54:26.048817 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:54:26.048817 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:54:26.050687 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:54:26.050687 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:26.050687 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:26.050687 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:26.050687 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:26.050687 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:26.050687 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:26.050687 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:26.050687 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:26.050687 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:26.050687 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:26.050687 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:26.050687 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:26.050687 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:26.050687 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:26.050687 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:26.050687 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:26.051941 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:54:26.051941 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:54:26.051941 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:54:26.051941 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:54:26.051941 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 621/1224] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:54:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:54:26.057461 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8929ae26-75a5-4a7c-b45d-a4ce14ddad3a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:26.175370 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8929ae26-75a5-4a7c-b45d-a4ce14ddad3a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:26.216842 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506bab45-fc90-497b-b5af-c79e316372ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:26.217480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bab45-fc90-497b-b5af-c79e316372ae None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:26.217808 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bab45-fc90-497b-b5af-c79e316372ae None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:26.218090 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bab45-fc90-497b-b5af-c79e316372ae None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:26.218875 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506bab45-fc90-497b-b5af-c79e316372ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:26.219135 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506bab45-fc90-497b-b5af-c79e316372ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:26.220771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506bab45-fc90-497b-b5af-c79e316372ae None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:26.238209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 604/1225] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:54:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:54:26.239563 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109b37a5-85ea-4d78-b719-fe9c2a46d22d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:26.345830 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:26.354438 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109b37a5-85ea-4d78-b719-fe9c2a46d22d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:26.395622 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2983e53-1460-4e7b-9cb9-1137a38edf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:26.396430 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2983e53-1460-4e7b-9cb9-1137a38edf7f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:26.396710 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2983e53-1460-4e7b-9cb9-1137a38edf7f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:26.396983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2983e53-1460-4e7b-9cb9-1137a38edf7f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:26.407044 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2983e53-1460-4e7b-9cb9-1137a38edf7f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:26.408520 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2983e53-1460-4e7b-9cb9-1137a38edf7f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:26.425075 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-a2983e53-1460-4e7b-9cb9-1137a38edf7f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 15:54:26.425075 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:26.425075 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:26.425075 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:26.425075 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:26.425075 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:26.425075 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:26.425075 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:26.425075 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:26.425075 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:26.425075 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:26.425075 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:26.425075 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:54:26.425075 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:54:26.425075 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:54:26.425075 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:54:26.426422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:26.426422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:26.426422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:54:26.426422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:54:26.426422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:54:26.426422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:54:26.426422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:26.426422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:26.426422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:26.426422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:26.426422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:26.426422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:26.426422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:26.426422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:26.426422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:26.426422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:26.426422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:26.426422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:26.427767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:26.427767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:26.427767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:54:26.427767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:54:26.427767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:54:26.427767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:54:26.427767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:54:26.427767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:54:26.427767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 15:54:26.427767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:54:26.427767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 622/1226] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:54:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:54:26.433046 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506bab45-fc90-497b-b5af-c79e316372ae None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:26.488653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:26.551737 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506bab45-fc90-497b-b5af-c79e316372ae None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:26.590647 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7fa67e-d173-4ac9-9ade-626f0d58b9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:26.591352 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7fa67e-d173-4ac9-9ade-626f0d58b9b6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:26.591637 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7fa67e-d173-4ac9-9ade-626f0d58b9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:26.591942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7fa67e-d173-4ac9-9ade-626f0d58b9b6 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:26.592950 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7fa67e-d173-4ac9-9ade-626f0d58b9b6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:26.593304 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7fa67e-d173-4ac9-9ade-626f0d58b9b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:26.595070 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7fa67e-d173-4ac9-9ade-626f0d58b9b6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:26.618063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 605/1227] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Jul 21 15:54:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:26.623672 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2983e53-1460-4e7b-9cb9-1137a38edf7f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:26.733533 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2983e53-1460-4e7b-9cb9-1137a38edf7f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:26.774086 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa528b6-49d6-482b-926f-2e45f76f937c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:26.774934 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa528b6-49d6-482b-926f-2e45f76f937c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:26.775217 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa528b6-49d6-482b-926f-2e45f76f937c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:26.775593 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa528b6-49d6-482b-926f-2e45f76f937c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:26.792016 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa528b6-49d6-482b-926f-2e45f76f937c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:26.793645 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa528b6-49d6-482b-926f-2e45f76f937c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:26.811942 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-4aa528b6-49d6-482b-926f-2e45f76f937c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:54:26.811942 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:26.811942 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:26.811942 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:26.811942 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:26.811942 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:26.811942 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:26.811942 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:26.811942 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:26.811942 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:26.811942 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:26.811942 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:26.811942 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:54:26.811942 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:54:26.811942 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:54:26.818156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:54:26.818156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:26.818156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:26.818156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:26.818156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:26.818156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:26.818156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:26.818156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:26.818156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:26.818156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:26.818156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:26.818156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:26.818156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:26.818156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:26.818156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:26.818156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:26.818156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:26.819289 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:54:26.819289 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:54:26.819289 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:54:26.819289 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:54:26.819289 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 623/1228] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:54:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:54:26.819670 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7fa67e-d173-4ac9-9ade-626f0d58b9b6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:26.957037 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7fa67e-d173-4ac9-9ade-626f0d58b9b6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:26.959632 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:26.997923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d59485-30df-44b9-b692-8933fd349cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:26.999012 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d59485-30df-44b9-b692-8933fd349cbf None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:26.999416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d59485-30df-44b9-b692-8933fd349cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:26.999893 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d59485-30df-44b9-b692-8933fd349cbf None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:27.001139 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d59485-30df-44b9-b692-8933fd349cbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:27.001699 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d59485-30df-44b9-b692-8933fd349cbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:27.003426 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d59485-30df-44b9-b692-8933fd349cbf None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:27.020488 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 606/1229] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:54:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:54:27.021880 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa528b6-49d6-482b-926f-2e45f76f937c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:27.032785 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:27.155022 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa528b6-49d6-482b-926f-2e45f76f937c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:27.191900 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc4a33-14a7-4aa9-86e0-dc2b7a4476c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:27.193699 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc4a33-14a7-4aa9-86e0-dc2b7a4476c5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:27.193991 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc4a33-14a7-4aa9-86e0-dc2b7a4476c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:27.194291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc4a33-14a7-4aa9-86e0-dc2b7a4476c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:27.202332 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc4a33-14a7-4aa9-86e0-dc2b7a4476c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:27.203820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc4a33-14a7-4aa9-86e0-dc2b7a4476c5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:27.219530 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-29dc4a33-14a7-4aa9-86e0-dc2b7a4476c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:54:27.219530 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:27.219530 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:27.219530 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:27.219530 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:27.219530 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:27.219530 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:27.219530 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:27.219530 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:27.219530 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:27.219530 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:27.219530 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:27.219530 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:54:27.219530 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:54:27.219530 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:54:27.220815 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:54:27.220815 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:27.220815 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:27.220815 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:27.220815 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:27.220815 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:27.220815 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:27.220815 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:27.220815 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:27.220815 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:27.220815 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:27.220815 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:27.220815 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:27.220815 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:27.220815 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:27.220815 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:27.220815 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:27.226698 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:54:27.226698 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:54:27.226698 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:54:27.226698 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:54:27.226698 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:54:27.226698 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:54:27.226698 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:54:27.226698 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:54:27.226698 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 624/1230] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:54:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:54:27.227208 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d59485-30df-44b9-b692-8933fd349cbf None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:27.345948 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d59485-30df-44b9-b692-8933fd349cbf None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:27.386215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb355d-6ebf-46f9-899c-c07f23278c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:27.386983 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb355d-6ebf-46f9-899c-c07f23278c70 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:27.387308 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb355d-6ebf-46f9-899c-c07f23278c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:27.387577 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb355d-6ebf-46f9-899c-c07f23278c70 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:27.388493 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfb355d-6ebf-46f9-899c-c07f23278c70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:27.388887 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfb355d-6ebf-46f9-899c-c07f23278c70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:27.390538 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfb355d-6ebf-46f9-899c-c07f23278c70 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:27.405103 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 607/1231] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:54:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:27.412141 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc4a33-14a7-4aa9-86e0-dc2b7a4476c5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:27.500757 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:27.546717 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc4a33-14a7-4aa9-86e0-dc2b7a4476c5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:27.584753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5175f-9bae-4d35-8e8f-c5594d8f650e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:27.585712 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5175f-9bae-4d35-8e8f-c5594d8f650e None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:27.585998 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5175f-9bae-4d35-8e8f-c5594d8f650e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:27.586281 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5175f-9bae-4d35-8e8f-c5594d8f650e None admin] PATH_INFO: `/v3/projects/bb1ac00308714c39a79aae09fbb73021/users/822d9d3521db4a9688f51c6313c7dd74/roles/64d94d0d5a7640818c21aa5a622624ab` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:27.603494 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5175f-9bae-4d35-8e8f-c5594d8f650e None admin] RBAC: Authorizing `identity:create_grant(project_id=bb1ac00308714c39a79aae09fbb73021, user_id=822d9d3521db4a9688f51c6313c7dd74, role_id=64d94d0d5a7640818c21aa5a622624ab)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:27.605146 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5175f-9bae-4d35-8e8f-c5594d8f650e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:27.621022 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 625/1232] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 15:54:27 2022] PUT /identity/v3/projects/bb1ac00308714c39a79aae09fbb73021/users/822d9d3521db4a9688f51c6313c7dd74/roles/64d94d0d5a7640818c21aa5a622624ab => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 15:54:29.789875 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cff85-3299-40ad-b9e1-8fc97250ca1c None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:29.790459 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cff85-3299-40ad-b9e1-8fc97250ca1c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:29.790459 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cff85-3299-40ad-b9e1-8fc97250ca1c None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:29.791944 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 608/1233] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:54:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 15:54:29.797779 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd5374-4377-492b-a860-e817e066fc92 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:29.798306 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd5374-4377-492b-a860-e817e066fc92 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:29.798404 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd5374-4377-492b-a860-e817e066fc92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:29.816135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-83fd5374-4377-492b-a860-e817e066fc92 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:54:29.817662 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-83fd5374-4377-492b-a860-e817e066fc92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:54:29.905895 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 626/1234] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:54:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:54:29.930568 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999cff85-3299-40ad-b9e1-8fc97250ca1c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:29.962695 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:30.048858 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999cff85-3299-40ad-b9e1-8fc97250ca1c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:30.084309 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93930ddb-168b-4ec2-b519-4a19d25ed18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:30.085137 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93930ddb-168b-4ec2-b519-4a19d25ed18f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:30.085423 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93930ddb-168b-4ec2-b519-4a19d25ed18f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:30.085732 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93930ddb-168b-4ec2-b519-4a19d25ed18f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:30.097737 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93930ddb-168b-4ec2-b519-4a19d25ed18f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:54:30.106190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-93930ddb-168b-4ec2-b519-4a19d25ed18f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:54:30.106190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:30.106190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:30.106190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:30.106190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:30.106190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:30.106190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:30.106190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:30.106190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:30.106190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:30.106190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:30.106190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:30.106190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:54:30.106190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:54:30.106190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:54:30.106190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:54:30.107445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:54:30.107445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:54:30.107445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:54:30.107445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:54:30.107445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:30.107445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:30.107445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:30.107445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:30.107445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:30.107445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:30.107445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:30.107445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:30.107445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:30.107445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:30.107445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:30.107445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:30.108883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:30.108883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:30.108883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:30.108883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:30.108883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:54:30.108883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:54:30.108883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:54:30.108883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:54:30.108883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:54:30.108883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:54:30.108883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 15:54:30.108883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:54:30.108883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 609/1235] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 15:54:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:54:30.114191 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd5374-4377-492b-a860-e817e066fc92 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:30.171740 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:30.234863 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd5374-4377-492b-a860-e817e066fc92 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:30.274375 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed8073b-0c21-46cf-a000-8822faa9601c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:30.275061 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8073b-0c21-46cf-a000-8822faa9601c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:30.275346 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8073b-0c21-46cf-a000-8822faa9601c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:30.276165 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8073b-0c21-46cf-a000-8822faa9601c None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:30.276565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed8073b-0c21-46cf-a000-8822faa9601c None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:30.276882 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed8073b-0c21-46cf-a000-8822faa9601c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:30.282444 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed8073b-0c21-46cf-a000-8822faa9601c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:30.287338 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 627/1236] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 15:54:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:30.294317 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93930ddb-168b-4ec2-b519-4a19d25ed18f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:30.398380 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:30.414770 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93930ddb-168b-4ec2-b519-4a19d25ed18f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:30.451804 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705fd85-52e4-4f1a-bc42-212b3e2c3aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:30.452700 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705fd85-52e4-4f1a-bc42-212b3e2c3aaf None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:30.452989 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705fd85-52e4-4f1a-bc42-212b3e2c3aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:30.453376 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705fd85-52e4-4f1a-bc42-212b3e2c3aaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:30.467589 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705fd85-52e4-4f1a-bc42-212b3e2c3aaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:30.469333 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705fd85-52e4-4f1a-bc42-212b3e2c3aaf None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:30.490102 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-1705fd85-52e4-4f1a-bc42-212b3e2c3aaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:54:30.490102 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:30.490102 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:30.490102 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:30.490102 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:30.490102 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:30.490102 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:30.490102 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:30.490102 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:30.490102 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:30.490102 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:30.490102 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:30.490102 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:54:30.490102 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:54:30.490102 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:54:30.492727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:54:30.492727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:30.492727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:30.492727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:30.492727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:30.492727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:30.492727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:30.492727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:30.492727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:30.492727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:30.492727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:30.492727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:30.492727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:30.492727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:30.492727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:30.492727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:30.492727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:30.494547 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:54:30.494547 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:54:30.494547 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:54:30.494547 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:54:30.494547 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 610/1237] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:54:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:54:30.498994 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed8073b-0c21-46cf-a000-8822faa9601c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:30.607380 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed8073b-0c21-46cf-a000-8822faa9601c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:30.617749 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:30.644937 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e17f3e-90ec-4685-980a-4e03bdb7d009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:30.645776 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e17f3e-90ec-4685-980a-4e03bdb7d009 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:30.646048 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e17f3e-90ec-4685-980a-4e03bdb7d009 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:30.646331 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e17f3e-90ec-4685-980a-4e03bdb7d009 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:30.647174 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e17f3e-90ec-4685-980a-4e03bdb7d009 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:30.647585 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e17f3e-90ec-4685-980a-4e03bdb7d009 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:30.649131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e17f3e-90ec-4685-980a-4e03bdb7d009 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:30.661679 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 628/1238] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:30.667728 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705fd85-52e4-4f1a-bc42-212b3e2c3aaf None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:30.786460 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705fd85-52e4-4f1a-bc42-212b3e2c3aaf None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:30.832093 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb8969-7899-4d30-9ebf-961f1ca92565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:30.834197 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb8969-7899-4d30-9ebf-961f1ca92565 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:30.834481 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb8969-7899-4d30-9ebf-961f1ca92565 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:30.834796 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb8969-7899-4d30-9ebf-961f1ca92565 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:30.840538 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:30.847680 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb8969-7899-4d30-9ebf-961f1ca92565 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:30.849986 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb8969-7899-4d30-9ebf-961f1ca92565 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:30.870721 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-9feb8969-7899-4d30-9ebf-961f1ca92565 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 15:54:30.870721 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:30.870721 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:30.870721 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:30.870721 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:30.870721 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:30.870721 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:30.870721 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:30.870721 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:30.870721 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:30.870721 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:30.870721 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:30.870721 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:54:30.870721 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:54:30.870721 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:54:30.870721 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:54:30.872125 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:30.872125 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:30.872125 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:54:30.872125 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:54:30.872125 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:54:30.872125 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:54:30.872125 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:30.872125 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:30.872125 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:30.872125 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:30.872125 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:30.872125 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:30.872125 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:30.872125 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:30.872125 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:30.872125 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:30.872125 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:30.872125 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:30.873382 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:30.873382 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:30.873382 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:54:30.873382 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:54:30.873382 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:54:30.873382 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:54:30.873382 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:54:30.873382 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:54:30.873382 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 15:54:30.873382 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:54:30.873382 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 611/1239] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:54:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:54:30.879074 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e17f3e-90ec-4685-980a-4e03bdb7d009 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:30.997762 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e17f3e-90ec-4685-980a-4e03bdb7d009 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:31.032422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab25527-e305-483b-a855-9228589e9224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:31.033086 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab25527-e305-483b-a855-9228589e9224 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:31.033368 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab25527-e305-483b-a855-9228589e9224 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:31.033674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab25527-e305-483b-a855-9228589e9224 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:31.034513 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab25527-e305-483b-a855-9228589e9224 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:31.034822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab25527-e305-483b-a855-9228589e9224 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:31.036272 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab25527-e305-483b-a855-9228589e9224 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:31.055356 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 629/1240] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Jul 21 15:54:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:31.062441 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb8969-7899-4d30-9ebf-961f1ca92565 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:31.176579 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb8969-7899-4d30-9ebf-961f1ca92565 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:31.218283 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d89468-e8b1-42dd-a497-74fe712f7334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:31.218935 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d89468-e8b1-42dd-a497-74fe712f7334 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:31.219280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d89468-e8b1-42dd-a497-74fe712f7334 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:31.219602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d89468-e8b1-42dd-a497-74fe712f7334 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:31.293556 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:31.301187 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d89468-e8b1-42dd-a497-74fe712f7334 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:31.302684 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d89468-e8b1-42dd-a497-74fe712f7334 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:31.409277 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 612/1241] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:54:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 15:54:31.415854 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab25527-e305-483b-a855-9228589e9224 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:31.444094 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:31.529194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab25527-e305-483b-a855-9228589e9224 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:31.567140 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d30c8c-ea83-47d9-9746-0a477f9442b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:31.567960 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d30c8c-ea83-47d9-9746-0a477f9442b8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:31.568279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d30c8c-ea83-47d9-9746-0a477f9442b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:31.568601 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d30c8c-ea83-47d9-9746-0a477f9442b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:31.579340 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d30c8c-ea83-47d9-9746-0a477f9442b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:31.580783 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d30c8c-ea83-47d9-9746-0a477f9442b8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:31.599186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-38d30c8c-ea83-47d9-9746-0a477f9442b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:54:31.599186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:31.599186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:31.599186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:31.599186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:31.599186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:31.599186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:31.599186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:31.599186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:31.599186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:31.599186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:31.599186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:31.599186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:54:31.599186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:54:31.599186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:54:31.600317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:54:31.600317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:31.600317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:31.600317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:31.600317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:31.600317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:31.600317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:31.600317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:31.600317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:31.600317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:31.600317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:31.600317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:31.600317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:31.600317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:31.600317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:31.600317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:31.600317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:31.602289 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:54:31.602289 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:54:31.602289 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:54:31.602289 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:54:31.602289 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 630/1242] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:54:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:54:31.607169 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d89468-e8b1-42dd-a497-74fe712f7334 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:31.704747 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:31.711576 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d89468-e8b1-42dd-a497-74fe712f7334 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:31.754235 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e5062c-eea1-4497-a3ed-38f2f6368c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:31.755022 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e5062c-eea1-4497-a3ed-38f2f6368c91 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:31.755305 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e5062c-eea1-4497-a3ed-38f2f6368c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:31.755587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e5062c-eea1-4497-a3ed-38f2f6368c91 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:31.756327 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e5062c-eea1-4497-a3ed-38f2f6368c91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:31.756646 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e5062c-eea1-4497-a3ed-38f2f6368c91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:31.758201 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e5062c-eea1-4497-a3ed-38f2f6368c91 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:31.775831 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 613/1243] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:54:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:31.777341 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d30c8c-ea83-47d9-9746-0a477f9442b8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:31.863544 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:31.890574 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d30c8c-ea83-47d9-9746-0a477f9442b8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:31.930119 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ea21b-e527-43e6-89e6-a4e055f4f72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:31.931072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ea21b-e527-43e6-89e6-a4e055f4f72c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:31.931442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ea21b-e527-43e6-89e6-a4e055f4f72c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:31.931799 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ea21b-e527-43e6-89e6-a4e055f4f72c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:31.939954 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ea21b-e527-43e6-89e6-a4e055f4f72c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:31.941674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ea21b-e527-43e6-89e6-a4e055f4f72c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:31.959381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-052ea21b-e527-43e6-89e6-a4e055f4f72c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:54:31.959381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:31.959381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:31.959381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:31.959381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:31.959381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:31.959381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:31.959381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:31.959381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:31.959381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:31.959381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:31.959381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:31.959381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:54:31.959381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:54:31.959381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:54:31.960962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:54:31.960962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:31.960962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:31.960962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:31.960962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:31.960962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:31.960962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:31.960962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:31.960962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:31.960962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:31.960962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:31.960962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:31.960962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:31.960962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:31.960962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:31.960962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:31.960962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:31.961999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:54:31.961999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:54:31.961999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:54:31.961999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:54:31.961999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:54:31.961999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:54:31.961999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:54:31.961999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:54:31.961999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 631/1244] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:54:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:54:31.968299 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e5062c-eea1-4497-a3ed-38f2f6368c91 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:32.100861 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e5062c-eea1-4497-a3ed-38f2f6368c91 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:32.143191 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced360e4-8261-42ec-96e3-829f7eefca23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:32.143776 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced360e4-8261-42ec-96e3-829f7eefca23 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:32.144064 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced360e4-8261-42ec-96e3-829f7eefca23 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:32.144343 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced360e4-8261-42ec-96e3-829f7eefca23 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:32.145143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced360e4-8261-42ec-96e3-829f7eefca23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:32.145457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced360e4-8261-42ec-96e3-829f7eefca23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:32.147059 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced360e4-8261-42ec-96e3-829f7eefca23 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:32.164989 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 614/1245] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:54:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:32.175550 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ea21b-e527-43e6-89e6-a4e055f4f72c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:32.318271 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ea21b-e527-43e6-89e6-a4e055f4f72c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:32.358499 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbdf671-1437-4503-a474-f860e8005f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:32.359339 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdf671-1437-4503-a474-f860e8005f0f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:32.360145 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdf671-1437-4503-a474-f860e8005f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:32.360560 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdf671-1437-4503-a474-f860e8005f0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:32.361494 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbdf671-1437-4503-a474-f860e8005f0f None admin] RBAC: Adding query filter params (role.id=64d94d0d5a7640818c21aa5a622624ab, scope.project.id=bb1ac00308714c39a79aae09fbb73021, user.id=822d9d3521db4a9688f51c6313c7dd74) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:32.361868 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbdf671-1437-4503-a474-f860e8005f0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:32.363435 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbdf671-1437-4503-a474-f860e8005f0f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:32.370802 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 632/1246] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:54:32 2022] GET /identity/v3/role_assignments?user.id=822d9d3521db4a9688f51c6313c7dd74&scope.project.id=bb1ac00308714c39a79aae09fbb73021&role.id=64d94d0d5a7640818c21aa5a622624ab => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:34.544904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6831c-54e4-4cc1-b711-08a4b0654ac7 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:34.545468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6831c-54e4-4cc1-b711-08a4b0654ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:34.545468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6831c-54e4-4cc1-b711-08a4b0654ac7 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:34.546975 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 615/1247] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:54:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 15:54:34.555706 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533f836-1dfb-4d6b-b04e-f1864006fdef None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:34.556192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533f836-1dfb-4d6b-b04e-f1864006fdef None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:34.556682 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533f836-1dfb-4d6b-b04e-f1864006fdef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:34.563386 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:34.577479 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1533f836-1dfb-4d6b-b04e-f1864006fdef None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:54:34.579014 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1533f836-1dfb-4d6b-b04e-f1864006fdef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:54:34.661415 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 633/1248] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:54:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:54:34.685137 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6831c-54e4-4cc1-b711-08a4b0654ac7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:34.704024 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:34.793297 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6831c-54e4-4cc1-b711-08a4b0654ac7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:34.830630 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee90a6-f0f1-4668-9727-64618caa7518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:34.831347 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee90a6-f0f1-4668-9727-64618caa7518 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:34.831631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee90a6-f0f1-4668-9727-64618caa7518 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:34.831902 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee90a6-f0f1-4668-9727-64618caa7518 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:34.842496 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ee90a6-f0f1-4668-9727-64618caa7518 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:54:34.850339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b4ee90a6-f0f1-4668-9727-64618caa7518 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:54:34.850339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:34.850339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:34.850339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:34.850339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:34.850339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:34.850339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:34.850339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:34.850339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:34.850339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:34.850339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:34.850339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:34.850339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:54:34.850339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:54:34.850339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:54:34.850339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:54:34.853813 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:54:34.853813 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:54:34.853813 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:54:34.853813 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:54:34.853813 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:34.853813 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:34.853813 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:34.853813 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:34.853813 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:34.853813 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:34.853813 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:34.853813 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:34.853813 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:34.853813 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:34.853813 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:34.853813 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:34.854753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:34.854753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:34.854753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:34.854753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:34.854753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:54:34.854753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:54:34.854753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:54:34.854753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:54:34.854753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:54:34.854753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:54:34.854753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:54:34.854753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:54:34.854753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 616/1249] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:54:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 15:54:34.861500 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533f836-1dfb-4d6b-b04e-f1864006fdef None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:34.975789 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533f836-1dfb-4d6b-b04e-f1864006fdef None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:35.011298 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:35.016796 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53296872-618a-4097-ab65-bbc68e4bec22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:35.017487 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53296872-618a-4097-ab65-bbc68e4bec22 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:35.017800 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53296872-618a-4097-ab65-bbc68e4bec22 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:35.018079 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53296872-618a-4097-ab65-bbc68e4bec22 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:35.018832 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53296872-618a-4097-ab65-bbc68e4bec22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:35.019185 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53296872-618a-4097-ab65-bbc68e4bec22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:35.020756 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53296872-618a-4097-ab65-bbc68e4bec22 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:35.029936 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 634/1250] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:54:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:35.036101 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee90a6-f0f1-4668-9727-64618caa7518 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:35.126258 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:35.164015 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee90a6-f0f1-4668-9727-64618caa7518 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:35.214153 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb08bc3-743a-4cc1-bcc4-48070ea01fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:35.215118 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb08bc3-743a-4cc1-bcc4-48070ea01fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:35.215434 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb08bc3-743a-4cc1-bcc4-48070ea01fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:35.215750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb08bc3-743a-4cc1-bcc4-48070ea01fb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:35.228082 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb08bc3-743a-4cc1-bcc4-48070ea01fb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:35.229544 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb08bc3-743a-4cc1-bcc4-48070ea01fb1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:35.252786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-deb08bc3-743a-4cc1-bcc4-48070ea01fb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:54:35.252786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:35.252786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:35.252786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:35.252786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:35.252786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:35.252786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:35.252786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:35.252786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:35.252786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:35.252786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:35.252786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:35.252786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:54:35.252786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:54:35.252786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:54:35.254128 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:54:35.254128 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:35.254128 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:35.254128 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:35.254128 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:35.254128 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:35.254128 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:35.254128 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:35.254128 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:35.254128 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:35.254128 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:35.254128 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:35.254128 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:35.254128 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:35.254128 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:35.254128 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:35.254128 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:35.255533 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:54:35.255533 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:54:35.255533 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:54:35.255533 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:54:35.255533 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 617/1251] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:54:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:54:35.261175 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53296872-618a-4097-ab65-bbc68e4bec22 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:35.379623 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53296872-618a-4097-ab65-bbc68e4bec22 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:35.416987 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108a7ea-e6c2-4ee8-91e6-25d0b869cd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:35.417768 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108a7ea-e6c2-4ee8-91e6-25d0b869cd37 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:35.418055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108a7ea-e6c2-4ee8-91e6-25d0b869cd37 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:35.418329 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108a7ea-e6c2-4ee8-91e6-25d0b869cd37 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:35.419094 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108a7ea-e6c2-4ee8-91e6-25d0b869cd37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:35.419401 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108a7ea-e6c2-4ee8-91e6-25d0b869cd37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:35.420830 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108a7ea-e6c2-4ee8-91e6-25d0b869cd37 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:35.435093 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 635/1252] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:54:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:35.442991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb08bc3-743a-4cc1-bcc4-48070ea01fb1 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:35.556663 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb08bc3-743a-4cc1-bcc4-48070ea01fb1 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:35.588562 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:35.601305 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852e607-79f3-4444-beff-f66509d2f6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:35.602511 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852e607-79f3-4444-beff-f66509d2f6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:35.602951 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852e607-79f3-4444-beff-f66509d2f6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:35.603379 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852e607-79f3-4444-beff-f66509d2f6c9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:35.613421 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1852e607-79f3-4444-beff-f66509d2f6c9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:35.615099 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1852e607-79f3-4444-beff-f66509d2f6c9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:35.630442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-1852e607-79f3-4444-beff-f66509d2f6c9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 15:54:35.630442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:35.630442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:35.630442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:35.630442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:35.630442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:35.630442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:35.630442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:35.630442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:35.630442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:35.630442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:35.630442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:35.630442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:54:35.630442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:54:35.630442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:54:35.630442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:54:35.631496 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:35.631496 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:35.631496 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:54:35.631496 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:54:35.631496 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:54:35.631496 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:54:35.631496 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:35.631496 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:35.631496 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:35.631496 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:35.631496 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:35.631496 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:35.631496 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:35.631496 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:35.631496 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:35.631496 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:35.631496 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:35.631496 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:35.632501 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:35.632501 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:35.632501 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:54:35.632501 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:54:35.632501 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:54:35.632501 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:54:35.632501 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:54:35.632501 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:54:35.632501 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 15:54:35.632501 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:54:35.632501 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 618/1253] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:54:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:54:35.637523 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108a7ea-e6c2-4ee8-91e6-25d0b869cd37 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:35.702268 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:35.755871 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108a7ea-e6c2-4ee8-91e6-25d0b869cd37 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:35.798953 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c66df9-fb16-41fd-8718-8739c02e00ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:35.800059 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c66df9-fb16-41fd-8718-8739c02e00ae None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:35.800528 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c66df9-fb16-41fd-8718-8739c02e00ae None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:35.800982 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c66df9-fb16-41fd-8718-8739c02e00ae None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:35.802465 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c66df9-fb16-41fd-8718-8739c02e00ae None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:35.802984 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c66df9-fb16-41fd-8718-8739c02e00ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:35.806002 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c66df9-fb16-41fd-8718-8739c02e00ae None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:35.830455 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 636/1254] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Jul 21 15:54:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:35.836732 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852e607-79f3-4444-beff-f66509d2f6c9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:35.969435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852e607-79f3-4444-beff-f66509d2f6c9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:36.006251 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a463a39e-adb1-43d9-88fb-a67825d8be3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:36.007476 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463a39e-adb1-43d9-88fb-a67825d8be3e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:36.007761 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463a39e-adb1-43d9-88fb-a67825d8be3e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:36.008043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463a39e-adb1-43d9-88fb-a67825d8be3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:36.067457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:36.082782 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a463a39e-adb1-43d9-88fb-a67825d8be3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:36.084258 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a463a39e-adb1-43d9-88fb-a67825d8be3e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:36.208208 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 619/1255] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:54:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 15:54:36.216744 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c66df9-fb16-41fd-8718-8739c02e00ae None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:36.344271 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c66df9-fb16-41fd-8718-8739c02e00ae None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:36.361011 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:36.392988 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ec9ea-51b6-4bf8-adea-887ed30d3974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:36.393909 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ec9ea-51b6-4bf8-adea-887ed30d3974 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:36.394271 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ec9ea-51b6-4bf8-adea-887ed30d3974 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:36.394557 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ec9ea-51b6-4bf8-adea-887ed30d3974 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:36.406654 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0ec9ea-51b6-4bf8-adea-887ed30d3974 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:36.408147 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0ec9ea-51b6-4bf8-adea-887ed30d3974 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:36.431695 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-2d0ec9ea-51b6-4bf8-adea-887ed30d3974 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:54:36.431695 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:36.431695 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:36.431695 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:36.431695 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:36.431695 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:36.431695 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:36.431695 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:36.431695 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:36.431695 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:36.431695 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:36.431695 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:36.431695 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:54:36.431695 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:54:36.431695 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:54:36.433031 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:54:36.433031 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:36.433031 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:36.433031 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:36.433031 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:36.433031 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:36.433031 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:36.433031 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:36.433031 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:36.433031 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:36.433031 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:36.433031 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:36.433031 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:36.433031 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:36.433031 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:36.433031 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:36.433031 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:36.434095 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:54:36.434095 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:54:36.434095 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:54:36.434095 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:54:36.434095 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 637/1256] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:54:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:54:36.440139 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a463a39e-adb1-43d9-88fb-a67825d8be3e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:36.551510 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:36.567305 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a463a39e-adb1-43d9-88fb-a67825d8be3e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:36.610106 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc8af8-a4ca-4ed4-af2c-9322db7f9223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:36.610920 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc8af8-a4ca-4ed4-af2c-9322db7f9223 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:36.611280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc8af8-a4ca-4ed4-af2c-9322db7f9223 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:36.611637 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc8af8-a4ca-4ed4-af2c-9322db7f9223 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:36.612438 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc8af8-a4ca-4ed4-af2c-9322db7f9223 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:36.612757 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc8af8-a4ca-4ed4-af2c-9322db7f9223 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:36.614183 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc8af8-a4ca-4ed4-af2c-9322db7f9223 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:36.628018 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 620/1257] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:54:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:36.634362 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ec9ea-51b6-4bf8-adea-887ed30d3974 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:36.745669 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ec9ea-51b6-4bf8-adea-887ed30d3974 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:36.792642 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3a04b-bf23-4ae5-943e-9c9d1e01d690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:36.793619 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3a04b-bf23-4ae5-943e-9c9d1e01d690 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:36.793916 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3a04b-bf23-4ae5-943e-9c9d1e01d690 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:36.794201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3a04b-bf23-4ae5-943e-9c9d1e01d690 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:36.803206 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f3a04b-bf23-4ae5-943e-9c9d1e01d690 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:36.804720 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f3a04b-bf23-4ae5-943e-9c9d1e01d690 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:36.822045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-58f3a04b-bf23-4ae5-943e-9c9d1e01d690 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:54:36.822045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:36.822045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:36.822045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:36.822045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:36.822045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:36.822045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:36.822045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:36.822045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:36.822045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:36.822045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:36.822045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:36.822045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:54:36.822045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:54:36.822045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:54:36.823078 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:54:36.823078 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:36.823078 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:36.823078 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:36.823078 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:36.823078 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:36.823078 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:36.823078 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:36.823078 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:36.823078 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:36.823078 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:36.823078 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:36.823078 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:36.823078 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:36.823078 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:36.823078 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:36.823078 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:36.824005 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:54:36.824005 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:54:36.824005 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:54:36.824005 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:54:36.824005 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:54:36.824005 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:54:36.824005 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:54:36.824005 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:54:36.824005 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 638/1258] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:54:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:54:36.829748 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc8af8-a4ca-4ed4-af2c-9322db7f9223 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:36.950304 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc8af8-a4ca-4ed4-af2c-9322db7f9223 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:36.988176 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb4546-e37e-4318-89c7-7e3f171cb04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:36.988902 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb4546-e37e-4318-89c7-7e3f171cb04d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:36.989188 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb4546-e37e-4318-89c7-7e3f171cb04d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:36.989466 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb4546-e37e-4318-89c7-7e3f171cb04d None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:36.990447 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb4546-e37e-4318-89c7-7e3f171cb04d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:36.990796 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb4546-e37e-4318-89c7-7e3f171cb04d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:36.992349 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb4546-e37e-4318-89c7-7e3f171cb04d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:37.006038 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 621/1259] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:54:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:37.012816 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3a04b-bf23-4ae5-943e-9c9d1e01d690 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:37.041877 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:37.121938 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3a04b-bf23-4ae5-943e-9c9d1e01d690 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:37.158708 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc7f39-6e67-402d-977b-da1b7f9e996f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:37.159371 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc7f39-6e67-402d-977b-da1b7f9e996f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:37.159757 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc7f39-6e67-402d-977b-da1b7f9e996f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:37.160018 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc7f39-6e67-402d-977b-da1b7f9e996f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:37.160949 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabc7f39-6e67-402d-977b-da1b7f9e996f None admin] RBAC: Adding query filter params (role.id=475521b8a0174037b162f92022aea1a1, scope.project.id=bb1ac00308714c39a79aae09fbb73021, user.id=822d9d3521db4a9688f51c6313c7dd74) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:37.161266 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabc7f39-6e67-402d-977b-da1b7f9e996f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:37.162760 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabc7f39-6e67-402d-977b-da1b7f9e996f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:37.168853 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 639/1260] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:54:37 2022] GET /identity/v3/role_assignments?user.id=822d9d3521db4a9688f51c6313c7dd74&scope.project.id=bb1ac00308714c39a79aae09fbb73021&role.id=475521b8a0174037b162f92022aea1a1 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:39.218033 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92190f3-2e1d-4cc2-9548-56ba3551695b None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:39.218611 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92190f3-2e1d-4cc2-9548-56ba3551695b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:39.218611 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92190f3-2e1d-4cc2-9548-56ba3551695b None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:39.219949 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 622/1261] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:54:39.226105 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e39666-f988-4293-b150-a5103113b931 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:39.226551 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e39666-f988-4293-b150-a5103113b931 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:39.226689 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e39666-f988-4293-b150-a5103113b931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:39.245391 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e4e39666-f988-4293-b150-a5103113b931 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:54:39.246627 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e4e39666-f988-4293-b150-a5103113b931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:54:39.314236 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 640/1262] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:54:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:54:39.336823 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92190f3-2e1d-4cc2-9548-56ba3551695b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:39.347905 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:39.438798 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92190f3-2e1d-4cc2-9548-56ba3551695b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:39.471766 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6ea41-2e52-4767-9bc8-d7631c794c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:39.472476 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6ea41-2e52-4767-9bc8-d7631c794c0e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:39.472758 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6ea41-2e52-4767-9bc8-d7631c794c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:39.473033 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6ea41-2e52-4767-9bc8-d7631c794c0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:39.483335 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe6ea41-2e52-4767-9bc8-d7631c794c0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:54:39.491400 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-dbe6ea41-2e52-4767-9bc8-d7631c794c0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:54:39.491400 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:39.491400 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:39.491400 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:39.491400 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:39.491400 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:39.491400 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:39.491400 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:39.491400 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:39.491400 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:39.491400 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:39.491400 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:39.491400 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:54:39.491400 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:54:39.491400 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:54:39.491400 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:54:39.492550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:54:39.492550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:54:39.492550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:54:39.492550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:54:39.492550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:39.492550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:39.492550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:39.492550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:39.492550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:39.492550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:39.492550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:39.492550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:39.492550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:39.492550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:39.492550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:39.492550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:39.493891 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:39.493891 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:39.493891 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:39.493891 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:39.493891 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:54:39.493891 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:54:39.493891 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:54:39.493891 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:54:39.493891 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:54:39.493891 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:54:39.493891 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:54:39.493891 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:54:39.493891 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 623/1263] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:54:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:54:39.498588 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e39666-f988-4293-b150-a5103113b931 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:39.516467 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:39.594730 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e39666-f988-4293-b150-a5103113b931 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:39.628018 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327df71-06d3-4008-9e62-3fe9f1ef6575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:39.628678 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327df71-06d3-4008-9e62-3fe9f1ef6575 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:39.628961 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327df71-06d3-4008-9e62-3fe9f1ef6575 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:39.629236 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327df71-06d3-4008-9e62-3fe9f1ef6575 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:39.630013 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7327df71-06d3-4008-9e62-3fe9f1ef6575 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:39.630322 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7327df71-06d3-4008-9e62-3fe9f1ef6575 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:39.631843 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7327df71-06d3-4008-9e62-3fe9f1ef6575 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:39.639811 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 641/1264] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:54:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:39.645690 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6ea41-2e52-4767-9bc8-d7631c794c0e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:39.715928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:39.750891 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6ea41-2e52-4767-9bc8-d7631c794c0e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:39.783773 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8bb2c-76f0-4475-a897-b2d72039f2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:39.784570 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8bb2c-76f0-4475-a897-b2d72039f2fa None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:39.784851 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8bb2c-76f0-4475-a897-b2d72039f2fa None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:39.785129 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8bb2c-76f0-4475-a897-b2d72039f2fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:39.796039 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a8bb2c-76f0-4475-a897-b2d72039f2fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:39.797489 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a8bb2c-76f0-4475-a897-b2d72039f2fa None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:39.814669 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-d3a8bb2c-76f0-4475-a897-b2d72039f2fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:54:39.814669 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:39.814669 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:39.814669 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:39.814669 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:39.814669 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:39.814669 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:39.814669 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:39.814669 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:39.814669 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:39.814669 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:39.814669 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:39.814669 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:54:39.814669 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:54:39.814669 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:54:39.815873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:54:39.815873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:39.815873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:39.815873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:39.815873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:39.815873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:39.815873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:39.815873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:39.815873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:39.815873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:39.815873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:39.815873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:39.815873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:39.815873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:39.815873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:39.815873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:39.815873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:39.817111 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:54:39.817111 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:54:39.817111 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:54:39.817111 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:54:39.817111 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 624/1265] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:54:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:54:39.821966 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327df71-06d3-4008-9e62-3fe9f1ef6575 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:39.918012 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:39.937155 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327df71-06d3-4008-9e62-3fe9f1ef6575 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:39.970814 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0611e3-1e1d-42da-9946-617a303b63ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:39.971480 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0611e3-1e1d-42da-9946-617a303b63ed None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:39.971757 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0611e3-1e1d-42da-9946-617a303b63ed None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:39.972035 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0611e3-1e1d-42da-9946-617a303b63ed None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:39.972757 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0611e3-1e1d-42da-9946-617a303b63ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:39.973065 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0611e3-1e1d-42da-9946-617a303b63ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:39.974475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0611e3-1e1d-42da-9946-617a303b63ed None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:39.986051 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 642/1266] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:54:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:39.992030 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8bb2c-76f0-4475-a897-b2d72039f2fa None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:40.096060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8bb2c-76f0-4475-a897-b2d72039f2fa None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:40.115833 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:40.131858 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40e549-fe5f-4a97-90fa-fa3a858d320d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:40.132673 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40e549-fe5f-4a97-90fa-fa3a858d320d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:40.132972 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40e549-fe5f-4a97-90fa-fa3a858d320d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:40.133249 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40e549-fe5f-4a97-90fa-fa3a858d320d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:40.142142 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad40e549-fe5f-4a97-90fa-fa3a858d320d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:40.145355 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad40e549-fe5f-4a97-90fa-fa3a858d320d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:40.164931 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ad40e549-fe5f-4a97-90fa-fa3a858d320d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 15:54:40.164931 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:40.164931 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:40.164931 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:40.164931 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:40.164931 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:40.164931 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:40.164931 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:40.164931 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:40.164931 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:40.164931 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:40.164931 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:40.164931 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:54:40.164931 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:54:40.164931 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:54:40.164931 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:54:40.166246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:40.166246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:40.166246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:54:40.166246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:54:40.166246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:54:40.166246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:54:40.166246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:40.166246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:40.166246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:40.166246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:40.166246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:40.166246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:40.166246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:40.166246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:40.166246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:40.166246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:40.166246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:40.166246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:40.167524 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:40.167524 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:40.167524 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:54:40.167524 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:54:40.167524 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:54:40.167524 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:54:40.167524 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:54:40.167524 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:54:40.167524 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 15:54:40.167524 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:54:40.167524 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 625/1267] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:54:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:54:40.173089 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0611e3-1e1d-42da-9946-617a303b63ed None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:40.278641 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0611e3-1e1d-42da-9946-617a303b63ed None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:40.318940 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455dba46-b82f-4f9e-8c4c-8216b9a29e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:40.320094 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455dba46-b82f-4f9e-8c4c-8216b9a29e21 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:40.320376 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455dba46-b82f-4f9e-8c4c-8216b9a29e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:40.320650 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455dba46-b82f-4f9e-8c4c-8216b9a29e21 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:40.321501 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455dba46-b82f-4f9e-8c4c-8216b9a29e21 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:40.321842 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455dba46-b82f-4f9e-8c4c-8216b9a29e21 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:40.323292 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455dba46-b82f-4f9e-8c4c-8216b9a29e21 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:40.332573 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:40.341560 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 643/1268] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Jul 21 15:54:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:40.347569 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40e549-fe5f-4a97-90fa-fa3a858d320d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:40.451395 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40e549-fe5f-4a97-90fa-fa3a858d320d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:40.485817 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a40f1-310a-4253-aece-84b4a0debf6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:40.486623 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a40f1-310a-4253-aece-84b4a0debf6e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:40.486907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a40f1-310a-4253-aece-84b4a0debf6e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:40.487184 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a40f1-310a-4253-aece-84b4a0debf6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:40.498410 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a40f1-310a-4253-aece-84b4a0debf6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:40.499869 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a40f1-310a-4253-aece-84b4a0debf6e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:40.517307 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-887a40f1-310a-4253-aece-84b4a0debf6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:54:40.517307 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:40.517307 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:40.517307 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:40.517307 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:40.517307 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:40.517307 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:40.517307 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:40.517307 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:40.517307 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:40.517307 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:40.517307 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:40.517307 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:54:40.517307 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:54:40.517307 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:54:40.518295 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:54:40.518295 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:40.518295 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:40.518295 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:40.518295 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:40.518295 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:40.518295 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:40.518295 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:40.518295 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:40.518295 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:40.518295 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:40.518295 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:40.518295 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:40.518295 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:40.518295 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:40.518295 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:40.518295 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:40.519265 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:54:40.519265 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:54:40.519265 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:54:40.519265 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:54:40.519265 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 626/1269] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:54:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:54:40.524406 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455dba46-b82f-4f9e-8c4c-8216b9a29e21 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:40.626052 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455dba46-b82f-4f9e-8c4c-8216b9a29e21 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:40.658965 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d95a47a-1d2f-4f89-bc31-6e591133d16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:40.659624 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95a47a-1d2f-4f89-bc31-6e591133d16d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:40.659904 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95a47a-1d2f-4f89-bc31-6e591133d16d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:40.660179 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95a47a-1d2f-4f89-bc31-6e591133d16d None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:40.660894 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d95a47a-1d2f-4f89-bc31-6e591133d16d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:40.661199 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d95a47a-1d2f-4f89-bc31-6e591133d16d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:40.663110 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d95a47a-1d2f-4f89-bc31-6e591133d16d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:40.674485 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 644/1270] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:40.680773 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a40f1-310a-4253-aece-84b4a0debf6e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:40.712100 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:40.790017 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a40f1-310a-4253-aece-84b4a0debf6e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:40.822597 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9052fd-915a-4741-a6bb-f0794f394c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:40.823380 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9052fd-915a-4741-a6bb-f0794f394c69 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:40.823663 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9052fd-915a-4741-a6bb-f0794f394c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:40.823938 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9052fd-915a-4741-a6bb-f0794f394c69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:40.830489 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9052fd-915a-4741-a6bb-f0794f394c69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:40.831926 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9052fd-915a-4741-a6bb-f0794f394c69 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:40.845952 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-7f9052fd-915a-4741-a6bb-f0794f394c69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:54:40.845952 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:40.845952 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:40.845952 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:40.845952 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:40.845952 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:40.845952 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:40.845952 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:40.845952 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:40.845952 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:40.845952 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:40.845952 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:40.845952 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:54:40.845952 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:54:40.845952 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:54:40.846904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:54:40.846904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:40.846904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:40.846904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:40.846904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:40.846904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:40.846904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:40.846904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:40.846904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:40.846904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:40.846904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:40.846904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:40.846904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:40.846904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:40.846904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:40.846904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:40.846904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:40.848063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:54:40.848063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:54:40.848063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:54:40.848063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:54:40.848063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:54:40.848063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:54:40.848063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:54:40.848063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:54:40.848063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 627/1271] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:54:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:54:40.854726 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d95a47a-1d2f-4f89-bc31-6e591133d16d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:40.922606 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:40.963363 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d95a47a-1d2f-4f89-bc31-6e591133d16d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:40.996202 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dda008-b5d7-4e69-9492-724806afb260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:40.996846 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dda008-b5d7-4e69-9492-724806afb260 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:40.997128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dda008-b5d7-4e69-9492-724806afb260 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:40.997401 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dda008-b5d7-4e69-9492-724806afb260 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:40.998208 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dda008-b5d7-4e69-9492-724806afb260 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:40.998517 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dda008-b5d7-4e69-9492-724806afb260 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:40.999936 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dda008-b5d7-4e69-9492-724806afb260 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:41.011374 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 645/1272] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:54:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:41.017337 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9052fd-915a-4741-a6bb-f0794f394c69 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:41.104716 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:41.113150 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9052fd-915a-4741-a6bb-f0794f394c69 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:41.146557 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a730181a-2293-4ad1-bdd1-14f32e809de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:41.147470 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a730181a-2293-4ad1-bdd1-14f32e809de6 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:41.147753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a730181a-2293-4ad1-bdd1-14f32e809de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:41.148028 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a730181a-2293-4ad1-bdd1-14f32e809de6 None admin] PATH_INFO: `/v3/projects/bb1ac00308714c39a79aae09fbb73021/users/822d9d3521db4a9688f51c6313c7dd74/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:41.149976 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a730181a-2293-4ad1-bdd1-14f32e809de6 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb1ac00308714c39a79aae09fbb73021, user_id=822d9d3521db4a9688f51c6313c7dd74, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:41.151403 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a730181a-2293-4ad1-bdd1-14f32e809de6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:41.163611 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 628/1273] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 15:54:41 2022] PUT /identity/v3/projects/bb1ac00308714c39a79aae09fbb73021/users/822d9d3521db4a9688f51c6313c7dd74/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 15:54:43.235763 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc08da-3cad-4a9a-b366-a0a522a18654 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:43.236286 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc08da-3cad-4a9a-b366-a0a522a18654 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:43.236373 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc08da-3cad-4a9a-b366-a0a522a18654 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:43.237722 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 646/1274] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:54:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:54:43.243706 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231ce75-1e8d-4f31-96c7-c205ed3d6865 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:43.244159 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231ce75-1e8d-4f31-96c7-c205ed3d6865 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:43.244259 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231ce75-1e8d-4f31-96c7-c205ed3d6865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:43.261223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e231ce75-1e8d-4f31-96c7-c205ed3d6865 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:54:43.262478 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e231ce75-1e8d-4f31-96c7-c205ed3d6865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:54:43.335184 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 629/1275] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:54:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:54:43.359353 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc08da-3cad-4a9a-b366-a0a522a18654 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:43.460711 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc08da-3cad-4a9a-b366-a0a522a18654 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:43.479157 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:43.497247 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930eff8a-9021-49b2-b444-44cdcd1e3b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:43.497984 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930eff8a-9021-49b2-b444-44cdcd1e3b8a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:43.498270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930eff8a-9021-49b2-b444-44cdcd1e3b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:43.498546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930eff8a-9021-49b2-b444-44cdcd1e3b8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:43.512284 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930eff8a-9021-49b2-b444-44cdcd1e3b8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 15:54:43.519873 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-930eff8a-9021-49b2-b444-44cdcd1e3b8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:54:43.519873 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:43.519873 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:43.519873 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:43.519873 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:43.519873 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:43.519873 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:43.519873 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:43.519873 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:43.519873 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:43.519873 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:43.519873 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:43.519873 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 15:54:43.519873 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 15:54:43.519873 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 15:54:43.519873 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 15:54:43.521004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 15:54:43.521004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 15:54:43.521004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 15:54:43.521004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 15:54:43.521004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:43.521004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:43.521004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:43.521004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:43.521004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:43.521004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:43.521004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:43.521004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:43.521004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:43.521004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:43.521004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:43.521004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:43.522163 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:43.522163 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:43.522163 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:43.522163 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:43.522163 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 15:54:43.522163 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 15:54:43.522163 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 15:54:43.522163 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 15:54:43.522163 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 15:54:43.522163 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 15:54:43.522163 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 15:54:43.522163 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:54:43.522163 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 647/1276] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 15:54:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:54:43.527238 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231ce75-1e8d-4f31-96c7-c205ed3d6865 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:43.609350 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:43.627617 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231ce75-1e8d-4f31-96c7-c205ed3d6865 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:43.659617 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a9270-561d-475f-b5c6-5b1bd2ceca62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:43.660271 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a9270-561d-475f-b5c6-5b1bd2ceca62 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:43.660554 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a9270-561d-475f-b5c6-5b1bd2ceca62 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:43.660830 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a9270-561d-475f-b5c6-5b1bd2ceca62 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:43.661600 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2a9270-561d-475f-b5c6-5b1bd2ceca62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:43.661910 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2a9270-561d-475f-b5c6-5b1bd2ceca62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:43.663972 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2a9270-561d-475f-b5c6-5b1bd2ceca62 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:43.671860 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 630/1277] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 15:54:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:43.677600 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930eff8a-9021-49b2-b444-44cdcd1e3b8a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:43.774321 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930eff8a-9021-49b2-b444-44cdcd1e3b8a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:43.807050 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d5729-6a3e-41c6-b761-419ab2fcfc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:43.807827 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d5729-6a3e-41c6-b761-419ab2fcfc07 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:43.808108 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d5729-6a3e-41c6-b761-419ab2fcfc07 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:43.808381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d5729-6a3e-41c6-b761-419ab2fcfc07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:43.818679 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d5729-6a3e-41c6-b761-419ab2fcfc07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:43.820146 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d5729-6a3e-41c6-b761-419ab2fcfc07 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:43.838072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-822d5729-6a3e-41c6-b761-419ab2fcfc07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:54:43.838072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:43.838072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:43.838072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:43.838072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:43.838072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:43.838072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:43.838072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:43.838072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:43.838072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:43.838072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:43.838072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:43.838072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:54:43.838072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:54:43.838072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:54:43.839193 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:54:43.839193 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:43.839193 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:43.839193 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:43.839193 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:43.839193 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:43.839193 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:43.839193 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:43.839193 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:43.839193 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:43.839193 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:43.839193 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:43.839193 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:43.839193 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:43.839193 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:43.839193 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:43.839193 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:43.840376 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:54:43.840376 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:54:43.840376 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:54:43.840376 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:54:43.840376 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 648/1278] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:54:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:54:43.845543 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a9270-561d-475f-b5c6-5b1bd2ceca62 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:43.955574 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a9270-561d-475f-b5c6-5b1bd2ceca62 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:43.988095 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f632a-f873-42ff-88fa-340987aa6f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:43.988748 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f632a-f873-42ff-88fa-340987aa6f7a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:43.989034 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f632a-f873-42ff-88fa-340987aa6f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:43.989314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f632a-f873-42ff-88fa-340987aa6f7a None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:43.990069 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f632a-f873-42ff-88fa-340987aa6f7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:43.990383 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f632a-f873-42ff-88fa-340987aa6f7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:43.992501 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f632a-f873-42ff-88fa-340987aa6f7a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:44.004505 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 631/1279] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:54:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:44.010204 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d5729-6a3e-41c6-b761-419ab2fcfc07 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:44.041364 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:44.109950 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d5729-6a3e-41c6-b761-419ab2fcfc07 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:44.142052 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483d7d0-5a98-404b-9445-66f651b9cb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:44.142836 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483d7d0-5a98-404b-9445-66f651b9cb26 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:44.143122 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483d7d0-5a98-404b-9445-66f651b9cb26 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:44.143401 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483d7d0-5a98-404b-9445-66f651b9cb26 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:44.151221 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d483d7d0-5a98-404b-9445-66f651b9cb26 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:44.152675 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d483d7d0-5a98-404b-9445-66f651b9cb26 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:44.167219 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d483d7d0-5a98-404b-9445-66f651b9cb26 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 15:54:44.167219 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:44.167219 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:44.167219 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:44.167219 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:44.167219 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:44.167219 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:44.167219 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:44.167219 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:44.167219 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:44.167219 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:44.167219 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:44.167219 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 15:54:44.167219 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 15:54:44.167219 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 15:54:44.167219 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 15:54:44.168368 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:44.168368 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:44.168368 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 15:54:44.168368 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:54:44.168368 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 15:54:44.168368 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 15:54:44.168368 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:44.168368 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:44.168368 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:44.168368 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:44.168368 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:44.168368 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:44.168368 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:44.168368 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:44.168368 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:44.168368 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:44.168368 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:44.168368 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:44.169775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:44.169775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:44.169775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 15:54:44.169775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 15:54:44.169775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 15:54:44.169775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 15:54:44.169775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 15:54:44.169775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 15:54:44.169775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 15:54:44.169775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:54:44.169775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 649/1280] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:54:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:54:44.174077 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f632a-f873-42ff-88fa-340987aa6f7a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:44.177788 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:44.276234 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f632a-f873-42ff-88fa-340987aa6f7a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:44.310332 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca4566-9991-4a9e-acfd-4c064739de86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:44.310930 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca4566-9991-4a9e-acfd-4c064739de86 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:44.311222 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca4566-9991-4a9e-acfd-4c064739de86 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:44.311501 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca4566-9991-4a9e-acfd-4c064739de86 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:44.312338 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ca4566-9991-4a9e-acfd-4c064739de86 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:44.312653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ca4566-9991-4a9e-acfd-4c064739de86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:44.314139 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ca4566-9991-4a9e-acfd-4c064739de86 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:44.330638 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 632/1281] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Jul 21 15:54:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:44.336993 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483d7d0-5a98-404b-9445-66f651b9cb26 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:44.426628 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:44.436094 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483d7d0-5a98-404b-9445-66f651b9cb26 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:44.468596 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aafe239-3511-4e31-accf-7bf94048c5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:44.469216 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aafe239-3511-4e31-accf-7bf94048c5e1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:44.469498 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aafe239-3511-4e31-accf-7bf94048c5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:44.469799 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aafe239-3511-4e31-accf-7bf94048c5e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:44.533655 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aafe239-3511-4e31-accf-7bf94048c5e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:44.535095 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aafe239-3511-4e31-accf-7bf94048c5e1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:44.632029 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:44.643728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 650/1282] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 15:54:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 15:54:44.650355 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca4566-9991-4a9e-acfd-4c064739de86 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:44.714630 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:44.754752 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca4566-9991-4a9e-acfd-4c064739de86 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:44.789931 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d3964-14bd-4dd1-80ec-d837f4e7aed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:44.790717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d3964-14bd-4dd1-80ec-d837f4e7aed3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:44.790999 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d3964-14bd-4dd1-80ec-d837f4e7aed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:44.792620 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d3964-14bd-4dd1-80ec-d837f4e7aed3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:44.801626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d3964-14bd-4dd1-80ec-d837f4e7aed3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:44.803598 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d3964-14bd-4dd1-80ec-d837f4e7aed3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:44.821492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-5b9d3964-14bd-4dd1-80ec-d837f4e7aed3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:54:44.821492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:44.821492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:44.821492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:44.821492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:44.821492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:44.821492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:44.821492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:44.821492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:44.821492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:44.821492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:44.821492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:44.821492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 15:54:44.821492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 15:54:44.821492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 15:54:44.822621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 15:54:44.822621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:44.822621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:44.822621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:44.822621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:44.822621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:44.822621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:44.822621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:44.822621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:44.822621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:44.822621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:44.822621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:44.822621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:44.822621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:44.822621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:44.822621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:44.822621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:44.823863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 15:54:44.823863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 15:54:44.823863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 15:54:44.823863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:54:44.823863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 633/1283] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:54:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:54:44.828695 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aafe239-3511-4e31-accf-7bf94048c5e1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:44.937609 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aafe239-3511-4e31-accf-7bf94048c5e1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:44.972032 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec602208-a981-438e-a367-bf0d18da6c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:44.972690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec602208-a981-438e-a367-bf0d18da6c20 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:44.972971 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec602208-a981-438e-a367-bf0d18da6c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:44.973245 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec602208-a981-438e-a367-bf0d18da6c20 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:44.973994 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec602208-a981-438e-a367-bf0d18da6c20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:44.974302 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec602208-a981-438e-a367-bf0d18da6c20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:44.975676 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec602208-a981-438e-a367-bf0d18da6c20 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:44.986723 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 651/1284] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 15:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:44.992923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d3964-14bd-4dd1-80ec-d837f4e7aed3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:45.097088 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d3964-14bd-4dd1-80ec-d837f4e7aed3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:45.111827 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:45.132469 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a7e98-0229-42f0-b6bf-79f7ce9e4338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:45.133255 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a7e98-0229-42f0-b6bf-79f7ce9e4338 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:45.133540 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a7e98-0229-42f0-b6bf-79f7ce9e4338 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:45.133848 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a7e98-0229-42f0-b6bf-79f7ce9e4338 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:45.140518 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a7e98-0229-42f0-b6bf-79f7ce9e4338 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:45.142017 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a7e98-0229-42f0-b6bf-79f7ce9e4338 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:45.155744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-d51a7e98-0229-42f0-b6bf-79f7ce9e4338 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:54:45.155744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:45.155744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:45.155744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:45.155744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:45.155744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:45.155744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:45.155744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:45.155744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:45.155744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:45.155744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:45.155744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:45.155744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 15:54:45.155744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 15:54:45.155744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 15:54:45.156866 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 15:54:45.156866 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:45.156866 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:45.156866 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:45.156866 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:45.156866 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:45.156866 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:45.156866 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:45.156866 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:45.156866 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:45.156866 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:45.156866 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:45.156866 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:45.156866 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:45.156866 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:45.156866 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:45.156866 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:45.158060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 15:54:45.158060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 15:54:45.158060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 15:54:45.158060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 15:54:45.158060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 15:54:45.158060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 15:54:45.158060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 15:54:45.158060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 15:54:45.158060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 634/1285] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 15:54:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:54:45.163320 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec602208-a981-438e-a367-bf0d18da6c20 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:45.211973 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:45.262210 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec602208-a981-438e-a367-bf0d18da6c20 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:45.295479 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3b1b5f-2792-4f67-b51e-f29a132bdd49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:45.296134 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b1b5f-2792-4f67-b51e-f29a132bdd49 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:45.296420 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b1b5f-2792-4f67-b51e-f29a132bdd49 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:45.296699 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b1b5f-2792-4f67-b51e-f29a132bdd49 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:45.297486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3b1b5f-2792-4f67-b51e-f29a132bdd49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:45.297830 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3b1b5f-2792-4f67-b51e-f29a132bdd49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:45.299276 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3b1b5f-2792-4f67-b51e-f29a132bdd49 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:45.312379 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 652/1286] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 15:54:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:45.318365 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a7e98-0229-42f0-b6bf-79f7ce9e4338 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:45.422049 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a7e98-0229-42f0-b6bf-79f7ce9e4338 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:45.474665 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8980653-ac4a-409a-8429-59853a5b042b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:45.476230 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8980653-ac4a-409a-8429-59853a5b042b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:45.476734 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8980653-ac4a-409a-8429-59853a5b042b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:45.477229 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8980653-ac4a-409a-8429-59853a5b042b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:45.478908 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8980653-ac4a-409a-8429-59853a5b042b None admin] RBAC: Adding query filter params (role.id=475521b8a0174037b162f92022aea1a1, scope.project.id=bb1ac00308714c39a79aae09fbb73021, user.id=822d9d3521db4a9688f51c6313c7dd74) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:45.479474 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8980653-ac4a-409a-8429-59853a5b042b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:45.482144 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8980653-ac4a-409a-8429-59853a5b042b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:45.495683 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 635/1287] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 15:54:45 2022] GET /identity/v3/role_assignments?user.id=822d9d3521db4a9688f51c6313c7dd74&scope.project.id=bb1ac00308714c39a79aae09fbb73021&role.id=475521b8a0174037b162f92022aea1a1 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:47.610276 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45246856-9444-4af9-a1ba-23444602facc None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:47.610797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45246856-9444-4af9-a1ba-23444602facc None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:47.610797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45246856-9444-4af9-a1ba-23444602facc None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:47.612104 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 653/1288] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:54:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:54:47.617806 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ac233-69c7-442c-91e1-3bfebcbcde4c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:47.618238 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ac233-69c7-442c-91e1-3bfebcbcde4c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:47.618365 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ac233-69c7-442c-91e1-3bfebcbcde4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:47.634756 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-604ac233-69c7-442c-91e1-3bfebcbcde4c None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:54:47.635939 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-604ac233-69c7-442c-91e1-3bfebcbcde4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:54:47.664730 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:47.704835 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 636/1289] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:54:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:54:47.727854 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45246856-9444-4af9-a1ba-23444602facc None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:47.827700 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45246856-9444-4af9-a1ba-23444602facc None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:47.838776 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:47.867225 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8585857-98c5-4e29-b69c-9eaeea371f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:47.867969 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8585857-98c5-4e29-b69c-9eaeea371f7f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:47.868388 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8585857-98c5-4e29-b69c-9eaeea371f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:47.868613 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8585857-98c5-4e29-b69c-9eaeea371f7f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:47.869342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8585857-98c5-4e29-b69c-9eaeea371f7f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:47.870997 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8585857-98c5-4e29-b69c-9eaeea371f7f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:47.884194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-c8585857-98c5-4e29-b69c-9eaeea371f7f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 15:54:47.884194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:47.884194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 15:54:47.884194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 15:54:47.884194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 15:54:47.884194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 15:54:47.884194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 15:54:47.884194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:54:47.884194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 15:54:47.884194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:54:47.884194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 15:54:47.884194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:54:47.884194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:47.884194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:47.884194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:47.884194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:47.884194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:47.885390 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:47.885390 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:47.885390 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:47.885390 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:47.885390 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:47.885390 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:47.885390 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 15:54:47.885390 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 15:54:47.885390 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 15:54:47.885390 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 15:54:47.885390 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:47.885390 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:47.885390 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:47.885390 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:47.885390 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:47.885390 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:47.886457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:47.886457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:47.886457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:47.886457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:47.886457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:47.886457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:47.886457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:47.886457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:47.886457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:47.886457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:47.886457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 15:54:47.886457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:54:47.886457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 15:54:47.886457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:54:47.886457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 654/1290] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jul 21 15:54:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:54:47.892478 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ac233-69c7-442c-91e1-3bfebcbcde4c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:48.006235 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ac233-69c7-442c-91e1-3bfebcbcde4c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:48.053037 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e95ad-3a90-4882-87b6-07538d3e70df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:48.053727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e95ad-3a90-4882-87b6-07538d3e70df None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:48.054015 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e95ad-3a90-4882-87b6-07538d3e70df None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:48.054293 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e95ad-3a90-4882-87b6-07538d3e70df None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:48.055012 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e95ad-3a90-4882-87b6-07538d3e70df None admin] RBAC: Adding query filter params (name=placement) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:48.055322 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e95ad-3a90-4882-87b6-07538d3e70df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:48.056696 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e95ad-3a90-4882-87b6-07538d3e70df None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:48.061553 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 637/1291] 10.222.0.52 () {56 vars in 1116 bytes} [Thu Jul 21 15:54:47 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:48.067459 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8585857-98c5-4e29-b69c-9eaeea371f7f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:48.179618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8585857-98c5-4e29-b69c-9eaeea371f7f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:48.222433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b8613f-274d-4a3c-94e4-54409feeb13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:48.223110 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8613f-274d-4a3c-94e4-54409feeb13b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:48.223399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8613f-274d-4a3c-94e4-54409feeb13b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:48.223678 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8613f-274d-4a3c-94e4-54409feeb13b None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:48.224392 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b8613f-274d-4a3c-94e4-54409feeb13b None admin] RBAC: Adding query filter params (type=placement) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:48.224737 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b8613f-274d-4a3c-94e4-54409feeb13b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:48.226176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b8613f-274d-4a3c-94e4-54409feeb13b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:48.230916 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 655/1292] 10.222.0.52 () {56 vars in 1116 bytes} [Thu Jul 21 15:54:48 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:50.262209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4debdf-4069-4663-ab67-74c3fe362027 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:50.262779 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4debdf-4069-4663-ab67-74c3fe362027 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:50.262779 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4debdf-4069-4663-ab67-74c3fe362027 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:50.264139 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 638/1293] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:54:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:54:50.269146 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300e731-522a-4f04-ab8d-cdfe17a11e86 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:50.269532 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300e731-522a-4f04-ab8d-cdfe17a11e86 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:50.269742 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300e731-522a-4f04-ab8d-cdfe17a11e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:50.286061 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a300e731-522a-4f04-ab8d-cdfe17a11e86 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:54:50.287259 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a300e731-522a-4f04-ab8d-cdfe17a11e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:54:50.315862 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:50.352400 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 656/1294] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:54:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:54:50.374697 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4debdf-4069-4663-ab67-74c3fe362027 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:50.406077 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:50.476646 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4debdf-4069-4663-ab67-74c3fe362027 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:50.512838 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba866e18-af8b-461a-aa81-c94231f48f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:50.513414 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba866e18-af8b-461a-aa81-c94231f48f9c None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:50.513733 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba866e18-af8b-461a-aa81-c94231f48f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:50.514017 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba866e18-af8b-461a-aa81-c94231f48f9c None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:50.514744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba866e18-af8b-461a-aa81-c94231f48f9c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:50.516133 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba866e18-af8b-461a-aa81-c94231f48f9c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:50.530066 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 639/1295] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 21 15:54:50 2022] POST /identity/v3/services => generated 246 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:54:52.631485 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3d5e6-21eb-4169-9551-753157401e2f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:52.632011 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3d5e6-21eb-4169-9551-753157401e2f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:52.632011 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3d5e6-21eb-4169-9551-753157401e2f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:52.633327 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 657/1296] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:54:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:54:52.639046 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb2699-2e1b-4751-98e5-91514c820858 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:52.639511 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb2699-2e1b-4751-98e5-91514c820858 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:52.639669 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb2699-2e1b-4751-98e5-91514c820858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:52.656703 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-afbb2699-2e1b-4751-98e5-91514c820858 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:54:52.657926 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-afbb2699-2e1b-4751-98e5-91514c820858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:54:52.729251 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 640/1297] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:54:52 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 15:54:52.751912 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3d5e6-21eb-4169-9551-753157401e2f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:52.860360 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3d5e6-21eb-4169-9551-753157401e2f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:52.901838 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec2c99b-10f5-4e43-bdeb-f466ba8fdc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:52.902438 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2c99b-10f5-4e43-bdeb-f466ba8fdc11 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:52.902728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2c99b-10f5-4e43-bdeb-f466ba8fdc11 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:52.903009 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2c99b-10f5-4e43-bdeb-f466ba8fdc11 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:52.903677 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec2c99b-10f5-4e43-bdeb-f466ba8fdc11 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:52.905072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec2c99b-10f5-4e43-bdeb-f466ba8fdc11 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:52.924151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-9ec2c99b-10f5-4e43-bdeb-f466ba8fdc11 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 15:54:52.924151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:52.924151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 15:54:52.924151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 15:54:52.924151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 15:54:52.924151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 15:54:52.924151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 15:54:52.924151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:54:52.924151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 15:54:52.924151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:54:52.924151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 15:54:52.924151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:54:52.924151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:52.924151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:52.924151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:52.924151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:52.924151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:52.926347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:52.926347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:52.926347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:52.926347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:52.926347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:52.926347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:52.926347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 15:54:52.926347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 15:54:52.926347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 15:54:52.926347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 15:54:52.926347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:52.926347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:52.926347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:52.926347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:52.926347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:52.926347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:52.928060 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:52.928060 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:52.928060 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:52.928060 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:52.928060 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:52.928060 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:52.928060 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:52.928060 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:52.928060 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:52.928060 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:52.928060 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 15:54:52.928060 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:54:52.928060 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 15:54:52.928060 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:54:52.928060 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 658/1298] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jul 21 15:54:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:54:52.933052 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb2699-2e1b-4751-98e5-91514c820858 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:52.954291 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:53.043813 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb2699-2e1b-4751-98e5-91514c820858 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:53.081377 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407ef9e-40e0-4a83-a23e-8f85eaea8bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:53.082168 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407ef9e-40e0-4a83-a23e-8f85eaea8bfc None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:53.082476 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407ef9e-40e0-4a83-a23e-8f85eaea8bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:53.082815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407ef9e-40e0-4a83-a23e-8f85eaea8bfc None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:53.083544 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407ef9e-40e0-4a83-a23e-8f85eaea8bfc None admin] RBAC: Adding query filter params (name=placement) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:53.083890 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407ef9e-40e0-4a83-a23e-8f85eaea8bfc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:53.085284 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407ef9e-40e0-4a83-a23e-8f85eaea8bfc None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:53.091099 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 641/1299] 10.222.0.52 () {56 vars in 1116 bytes} [Thu Jul 21 15:54:52 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 15:54:53.097702 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec2c99b-10f5-4e43-bdeb-f466ba8fdc11 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:53.129944 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:53.204241 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec2c99b-10f5-4e43-bdeb-f466ba8fdc11 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:53.244856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788f479-d347-4964-92cd-b6f46d43a6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:53.245513 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788f479-d347-4964-92cd-b6f46d43a6ad None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:53.245834 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788f479-d347-4964-92cd-b6f46d43a6ad None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:53.246111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788f479-d347-4964-92cd-b6f46d43a6ad None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:53.246906 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788f479-d347-4964-92cd-b6f46d43a6ad None admin] RBAC: Adding query filter params (interface=public, service_id=1516ec34a41747bf87c9f659a2627b07, region_id=RegionOne) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:53.247216 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788f479-d347-4964-92cd-b6f46d43a6ad None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:53.248591 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788f479-d347-4964-92cd-b6f46d43a6ad None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:53.253746 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 659/1300] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Jul 21 15:54:53 2022] GET /identity/v3/endpoints?service_id=1516ec34a41747bf87c9f659a2627b07&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:53.259841 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407ef9e-40e0-4a83-a23e-8f85eaea8bfc None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:53.341027 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:53.359439 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407ef9e-40e0-4a83-a23e-8f85eaea8bfc None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:53.392151 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645b43e-3c2d-4e95-9379-fc386a78bee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:53.392794 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645b43e-3c2d-4e95-9379-fc386a78bee4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:53.393075 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645b43e-3c2d-4e95-9379-fc386a78bee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:53.393349 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645b43e-3c2d-4e95-9379-fc386a78bee4 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:53.394059 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4645b43e-3c2d-4e95-9379-fc386a78bee4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:53.395414 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4645b43e-3c2d-4e95-9379-fc386a78bee4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:53.402109 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 642/1301] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jul 21 15:54:53 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 15:54:55.449049 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd58208-b748-4fbc-b446-896006dd25db None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:55.449602 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd58208-b748-4fbc-b446-896006dd25db None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:55.449727 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd58208-b748-4fbc-b446-896006dd25db None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:55.451058 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 660/1302] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:54:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:54:55.456517 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab4aa0-93f2-4b4f-870a-aade00a94c3b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:55.457144 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab4aa0-93f2-4b4f-870a-aade00a94c3b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:55.457144 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab4aa0-93f2-4b4f-870a-aade00a94c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:55.473698 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-17ab4aa0-93f2-4b4f-870a-aade00a94c3b None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:54:55.474882 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-17ab4aa0-93f2-4b4f-870a-aade00a94c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:54:55.539458 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 643/1303] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:54:55 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 15:54:55.561437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd58208-b748-4fbc-b446-896006dd25db None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:55.654267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:55.661169 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd58208-b748-4fbc-b446-896006dd25db None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:55.694299 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d853ee3-9d4a-4e39-8820-b2c8bc6224bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:55.695085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d853ee3-9d4a-4e39-8820-b2c8bc6224bb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:55.695368 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d853ee3-9d4a-4e39-8820-b2c8bc6224bb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:55.695641 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d853ee3-9d4a-4e39-8820-b2c8bc6224bb None admin] PATH_INFO: `/v3/services/placement` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:55.696288 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d853ee3-9d4a-4e39-8820-b2c8bc6224bb None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:55.697695 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d853ee3-9d4a-4e39-8820-b2c8bc6224bb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:55.709933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-0d853ee3-9d4a-4e39-8820-b2c8bc6224bb None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 15:54:55.709933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:55.709933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 15:54:55.709933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 15:54:55.709933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 15:54:55.709933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 15:54:55.709933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 15:54:55.709933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:54:55.709933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 15:54:55.709933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:54:55.709933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 15:54:55.709933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:54:55.709933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 15:54:55.709933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 15:54:55.709933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 15:54:55.709933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 15:54:55.709933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 15:54:55.711167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 15:54:55.711167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 15:54:55.711167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 15:54:55.711167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 15:54:55.711167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 15:54:55.711167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 15:54:55.711167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 15:54:55.711167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 15:54:55.711167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 15:54:55.711167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 15:54:55.711167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 15:54:55.711167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 15:54:55.711167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 15:54:55.711167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 15:54:55.711167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 15:54:55.711167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 15:54:55.712027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 15:54:55.712027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 15:54:55.712027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 15:54:55.712027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 15:54:55.712027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 15:54:55.712027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 15:54:55.712027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 15:54:55.712027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 15:54:55.712027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 15:54:55.712027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 15:54:55.712027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 15:54:55.712027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 15:54:55.712027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 15:54:55.712027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 15:54:55.712027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 661/1304] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jul 21 15:54:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 15:54:55.717147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab4aa0-93f2-4b4f-870a-aade00a94c3b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:55.815482 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:54:55.838184 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab4aa0-93f2-4b4f-870a-aade00a94c3b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:55.875215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61b0d4-c706-4175-99e5-1ccf7f3d94d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:55.876033 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61b0d4-c706-4175-99e5-1ccf7f3d94d9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:55.876371 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61b0d4-c706-4175-99e5-1ccf7f3d94d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:55.876678 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61b0d4-c706-4175-99e5-1ccf7f3d94d9 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:55.877631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc61b0d4-c706-4175-99e5-1ccf7f3d94d9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 15:54:55.878017 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc61b0d4-c706-4175-99e5-1ccf7f3d94d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:55.879481 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc61b0d4-c706-4175-99e5-1ccf7f3d94d9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:55.885278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 644/1305] 10.222.0.52 () {56 vars in 1116 bytes} [Thu Jul 21 15:54:55 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 15:54:55.891908 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d853ee3-9d4a-4e39-8820-b2c8bc6224bb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:54:55.989598 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d853ee3-9d4a-4e39-8820-b2c8bc6224bb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:54:56.022442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20b9926-3635-45e3-bd78-c97c3bc09ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:54:56.023084 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b9926-3635-45e3-bd78-c97c3bc09ead None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:54:56.023366 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b9926-3635-45e3-bd78-c97c3bc09ead None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:54:56.023640 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b9926-3635-45e3-bd78-c97c3bc09ead None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:54:56.024461 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20b9926-3635-45e3-bd78-c97c3bc09ead None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:54:56.025856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20b9926-3635-45e3-bd78-c97c3bc09ead None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:54:56.043932 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 662/1306] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jul 21 15:54:55 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 15:55:05.882297 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=30 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:55:06.039587 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:00.208005 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53645fb-23fd-40bc-ac8d-83959805e4fc None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:00.208559 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53645fb-23fd-40bc-ac8d-83959805e4fc None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:00.208651 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53645fb-23fd-40bc-ac8d-83959805e4fc None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:00.215150 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d76e9-7570-4216-8910-c96de9579a68 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:00.215595 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d76e9-7570-4216-8910-c96de9579a68 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:00.215687 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d76e9-7570-4216-8910-c96de9579a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:00.219185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 645/1307] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:56:00 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:56:00.302130 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9c6d76e9-7570-4216-8910-c96de9579a68 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:56:00.304132 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9c6d76e9-7570-4216-8910-c96de9579a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:00.442321 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 663/1308] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:56:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 15:56:00.510603 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7ee97-f556-43dc-b3d4-e7f4adde2364 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:00.510894 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7ee97-f556-43dc-b3d4-e7f4adde2364 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:00.511178 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7ee97-f556-43dc-b3d4-e7f4adde2364 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:00.512582 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 646/1309] 10.222.0.52 () {54 vars in 965 bytes} [Thu Jul 21 15:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:56:00.527142 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79424a2-995e-497d-81c9-43f05c73c788 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:00.527705 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79424a2-995e-497d-81c9-43f05c73c788 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:00.528193 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79424a2-995e-497d-81c9-43f05c73c788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:00.634836 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f79424a2-995e-497d-81c9-43f05c73c788 None None] MFA Rules not processed for user `1654fdf5ba6b46ba8d9b86a83a76a4b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:56:00.636059 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f79424a2-995e-497d-81c9-43f05c73c788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:00.658521 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:00.718741 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 664/1310] 10.222.0.52 () {58 vars in 1062 bytes} [Thu Jul 21 15:56:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:56:00.728325 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7ee97-f556-43dc-b3d4-e7f4adde2364 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:56:00.730075 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-68f7ee97-f556-43dc-b3d4-e7f4adde2364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:00.849561 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7ee97-f556-43dc-b3d4-e7f4adde2364 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:56:00.929361 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f1c52a-f242-4567-bff4-d29c2b4157df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:56:00.930396 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1c52a-f242-4567-bff4-d29c2b4157df service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:00.930853 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1c52a-f242-4567-bff4-d29c2b4157df service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:00.931295 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1c52a-f242-4567-bff4-d29c2b4157df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:00.933075 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e5f1c52a-f242-4567-bff4-d29c2b4157df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:00.990867 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:01.051001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f1c52a-f242-4567-bff4-d29c2b4157df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:56:01.052744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f1c52a-f242-4567-bff4-d29c2b4157df service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:56:01.072700 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 647/1311] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:56:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:56:03.415343 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd996e-7e3f-4cd5-ae0c-60af1765b14d None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:03.415944 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd996e-7e3f-4cd5-ae0c-60af1765b14d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:03.415944 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd996e-7e3f-4cd5-ae0c-60af1765b14d None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:03.417278 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 665/1312] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:56:03.422602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a748cd2-7bb9-4e8d-9c6c-263b36bebd50 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:03.423035 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a748cd2-7bb9-4e8d-9c6c-263b36bebd50 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:03.423170 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a748cd2-7bb9-4e8d-9c6c-263b36bebd50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:03.525113 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8a748cd2-7bb9-4e8d-9c6c-263b36bebd50 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:56:03.526353 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8a748cd2-7bb9-4e8d-9c6c-263b36bebd50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:03.638800 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 648/1313] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:56:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:56:03.678054 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd996e-7e3f-4cd5-ae0c-60af1765b14d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:56:03.679641 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-80bd996e-7e3f-4cd5-ae0c-60af1765b14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:03.802958 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd996e-7e3f-4cd5-ae0c-60af1765b14d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:56:03.888813 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:03.892168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a650739-90c3-4b02-9b96-1a9e524b0ba7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:56:03.892974 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a650739-90c3-4b02-9b96-1a9e524b0ba7 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:03.894133 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a650739-90c3-4b02-9b96-1a9e524b0ba7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:03.894583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a650739-90c3-4b02-9b96-1a9e524b0ba7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:03.897950 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1a650739-90c3-4b02-9b96-1a9e524b0ba7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:04.012845 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a650739-90c3-4b02-9b96-1a9e524b0ba7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:56:04.014418 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a650739-90c3-4b02-9b96-1a9e524b0ba7 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:56:04.034911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 666/1314] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:56:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 15:56:06.398983 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496a28d-220a-41de-9436-e70734d6ed6f None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:06.399535 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496a28d-220a-41de-9436-e70734d6ed6f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:06.399535 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496a28d-220a-41de-9436-e70734d6ed6f None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:06.400988 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 649/1315] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:56:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:56:06.406180 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506359b4-46b8-4707-b3c0-bb32574cb2a1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:06.406615 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506359b4-46b8-4707-b3c0-bb32574cb2a1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:06.406735 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506359b4-46b8-4707-b3c0-bb32574cb2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:06.470030 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-506359b4-46b8-4707-b3c0-bb32574cb2a1 None None] MFA Rules not processed for user `7c7ec8ec280649a8bf1f51745c94153b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:56:06.471253 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-506359b4-46b8-4707-b3c0-bb32574cb2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:06.527446 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:06.571679 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 667/1316] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:56:06 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:56:06.623990 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d612753-5046-4b5f-8ec7-1a0dc208be09 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:06.624307 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d612753-5046-4b5f-8ec7-1a0dc208be09 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:06.624585 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d612753-5046-4b5f-8ec7-1a0dc208be09 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:06.626010 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 650/1317] 10.222.0.52 () {54 vars in 965 bytes} [Thu Jul 21 15:56:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:56:06.632314 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0004eb-0597-40b8-bc18-b0589293f42b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:06.632731 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0004eb-0597-40b8-bc18-b0589293f42b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:06.633007 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0004eb-0597-40b8-bc18-b0589293f42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:06.705038 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9b0004eb-0597-40b8-bc18-b0589293f42b None None] MFA Rules not processed for user `1654fdf5ba6b46ba8d9b86a83a76a4b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:56:06.706408 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9b0004eb-0597-40b8-bc18-b0589293f42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:06.780786 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 668/1318] 10.222.0.52 () {58 vars in 1062 bytes} [Thu Jul 21 15:56:06 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 15:56:06.789727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d612753-5046-4b5f-8ec7-1a0dc208be09 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:56:06.791255 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0d612753-5046-4b5f-8ec7-1a0dc208be09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:06.801366 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:06.894199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d612753-5046-4b5f-8ec7-1a0dc208be09 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:56:06.965452 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb526ba-5faa-4d3a-8524-951d3d8fbf10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:56:06.966120 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb526ba-5faa-4d3a-8524-951d3d8fbf10 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:06.966408 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb526ba-5faa-4d3a-8524-951d3d8fbf10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:06.966690 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb526ba-5faa-4d3a-8524-951d3d8fbf10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:06.968331 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6cb526ba-5faa-4d3a-8524-951d3d8fbf10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:07.029051 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:07.074503 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb526ba-5faa-4d3a-8524-951d3d8fbf10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:56:07.075898 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb526ba-5faa-4d3a-8524-951d3d8fbf10 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:56:07.094744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 651/1319] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:56:06 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:56:10.073018 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0a288-38f7-4575-abc0-9afec960b565 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:10.073620 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0a288-38f7-4575-abc0-9afec960b565 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:10.073620 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0a288-38f7-4575-abc0-9afec960b565 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:10.075014 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 669/1320] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:56:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:56:10.080244 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901708e-bf4d-49e7-a8b9-a12ca09ad92c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:10.081197 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901708e-bf4d-49e7-a8b9-a12ca09ad92c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:10.081409 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901708e-bf4d-49e7-a8b9-a12ca09ad92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:10.145193 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0901708e-bf4d-49e7-a8b9-a12ca09ad92c None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:56:10.146443 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0901708e-bf4d-49e7-a8b9-a12ca09ad92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:10.244614 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 652/1321] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:56:10 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:56:10.284144 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf0a288-38f7-4575-abc0-9afec960b565 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:56:10.285707 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-acf0a288-38f7-4575-abc0-9afec960b565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:10.335959 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:10.389975 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf0a288-38f7-4575-abc0-9afec960b565 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:56:10.460795 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf3d2a8-85e7-4ae5-927f-2edf319133a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:56:10.461431 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3d2a8-85e7-4ae5-927f-2edf319133a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:10.461745 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3d2a8-85e7-4ae5-927f-2edf319133a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:10.462019 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3d2a8-85e7-4ae5-927f-2edf319133a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:10.463661 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6cf3d2a8-85e7-4ae5-927f-2edf319133a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:10.560434 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:10.574520 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf3d2a8-85e7-4ae5-927f-2edf319133a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:56:10.575976 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf3d2a8-85e7-4ae5-927f-2edf319133a5 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:56:10.593813 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 670/1322] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:56:10 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:56:13.404194 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54836ed-0742-4e4d-989b-7cb60289e5be None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:13.404753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54836ed-0742-4e4d-989b-7cb60289e5be None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:13.404753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54836ed-0742-4e4d-989b-7cb60289e5be None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:13.406359 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 653/1323] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:56:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:56:13.412302 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f5206-8fab-4bc6-9c97-5e1909979fad None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:13.412747 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f5206-8fab-4bc6-9c97-5e1909979fad None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:13.412864 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f5206-8fab-4bc6-9c97-5e1909979fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:13.478500 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0f8f5206-8fab-4bc6-9c97-5e1909979fad None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:56:13.479758 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0f8f5206-8fab-4bc6-9c97-5e1909979fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:13.586583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 671/1324] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:56:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:56:13.623978 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54836ed-0742-4e4d-989b-7cb60289e5be None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:56:13.625696 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a54836ed-0742-4e4d-989b-7cb60289e5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:13.646628 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:13.733928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54836ed-0742-4e4d-989b-7cb60289e5be None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:56:13.808233 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46605357-d1a9-4605-88bc-be0a2badd3fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:56:13.808882 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46605357-d1a9-4605-88bc-be0a2badd3fd service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:13.809165 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46605357-d1a9-4605-88bc-be0a2badd3fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:13.809443 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46605357-d1a9-4605-88bc-be0a2badd3fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:13.811120 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-46605357-d1a9-4605-88bc-be0a2badd3fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:13.885754 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:13.924776 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46605357-d1a9-4605-88bc-be0a2badd3fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:56:13.926257 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46605357-d1a9-4605-88bc-be0a2badd3fd service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:56:13.944831 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 654/1325] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:56:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:56:16.029414 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39f8c2-aea5-48e2-9ab8-10862aab69ad None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:16.029981 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39f8c2-aea5-48e2-9ab8-10862aab69ad None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:16.029981 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39f8c2-aea5-48e2-9ab8-10862aab69ad None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:16.031355 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 672/1326] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:56:16.036578 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856a8cf-0866-45de-80f3-27eb9d83d9d8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:16.036987 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856a8cf-0866-45de-80f3-27eb9d83d9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:16.037267 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856a8cf-0866-45de-80f3-27eb9d83d9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:16.101298 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9856a8cf-0866-45de-80f3-27eb9d83d9d8 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:56:16.102581 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9856a8cf-0866-45de-80f3-27eb9d83d9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:16.196589 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 655/1327] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:56:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:56:16.232509 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba39f8c2-aea5-48e2-9ab8-10862aab69ad None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:56:16.234085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ba39f8c2-aea5-48e2-9ab8-10862aab69ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:16.295546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:16.341290 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba39f8c2-aea5-48e2-9ab8-10862aab69ad None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:56:16.417897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc3d7b-f30f-4178-8e51-9e93653ee0cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:56:16.418545 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc3d7b-f30f-4178-8e51-9e93653ee0cc service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:16.418824 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc3d7b-f30f-4178-8e51-9e93653ee0cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:16.419099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc3d7b-f30f-4178-8e51-9e93653ee0cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:16.420737 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-60dc3d7b-f30f-4178-8e51-9e93653ee0cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:16.534940 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:16.537746 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc3d7b-f30f-4178-8e51-9e93653ee0cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:56:16.539753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc3d7b-f30f-4178-8e51-9e93653ee0cc service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:56:16.557791 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 673/1328] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:56:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:56:18.692149 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd011ce8-7c1a-4710-b3f9-d9af59be26f4 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:18.692613 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd011ce8-7c1a-4710-b3f9-d9af59be26f4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:18.692738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd011ce8-7c1a-4710-b3f9-d9af59be26f4 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:18.694132 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 656/1329] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:56:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:56:18.699649 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae07d7d-29f8-4f8e-9b13-ebf50c6afa82 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:18.700100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae07d7d-29f8-4f8e-9b13-ebf50c6afa82 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:18.700211 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae07d7d-29f8-4f8e-9b13-ebf50c6afa82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:18.772573 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-dae07d7d-29f8-4f8e-9b13-ebf50c6afa82 None None] MFA Rules not processed for user `7c7ec8ec280649a8bf1f51745c94153b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:56:18.773826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-dae07d7d-29f8-4f8e-9b13-ebf50c6afa82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:18.871850 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 674/1330] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:56:18 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:56:18.910144 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd011ce8-7c1a-4710-b3f9-d9af59be26f4 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:56:18.911722 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-dd011ce8-7c1a-4710-b3f9-d9af59be26f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:18.946394 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:19.026983 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd011ce8-7c1a-4710-b3f9-d9af59be26f4 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:56:19.100001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f4470-66a8-4df8-b6df-7dca28c4c7ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:56:19.100645 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f4470-66a8-4df8-b6df-7dca28c4c7ab service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:19.100928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f4470-66a8-4df8-b6df-7dca28c4c7ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:19.101200 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f4470-66a8-4df8-b6df-7dca28c4c7ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:19.102871 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f35f4470-66a8-4df8-b6df-7dca28c4c7ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:19.193553 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:19.227739 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35f4470-66a8-4df8-b6df-7dca28c4c7ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:56:19.229176 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35f4470-66a8-4df8-b6df-7dca28c4c7ab service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:56:19.249786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 657/1331] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:56:18 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:56:21.578847 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27390e-6282-4bf7-ab25-7b8ad81136e5 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:21.579355 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27390e-6282-4bf7-ab25-7b8ad81136e5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:21.579355 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27390e-6282-4bf7-ab25-7b8ad81136e5 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:21.580687 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 675/1332] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:56:21.586042 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4c568-4760-4461-940b-1f61f3ec77e5 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:21.586491 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4c568-4760-4461-940b-1f61f3ec77e5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:21.586604 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4c568-4760-4461-940b-1f61f3ec77e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:21.650409 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-73f4c568-4760-4461-940b-1f61f3ec77e5 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:56:21.651628 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-73f4c568-4760-4461-940b-1f61f3ec77e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:21.746802 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 658/1333] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:56:21.782584 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c27390e-6282-4bf7-ab25-7b8ad81136e5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:56:21.784717 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2c27390e-6282-4bf7-ab25-7b8ad81136e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:21.853077 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:21.889798 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c27390e-6282-4bf7-ab25-7b8ad81136e5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:56:21.967915 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0be5b6-f9ef-404b-8e2e-7715a157017e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:56:21.968569 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0be5b6-f9ef-404b-8e2e-7715a157017e service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:21.968856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0be5b6-f9ef-404b-8e2e-7715a157017e service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:21.969134 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0be5b6-f9ef-404b-8e2e-7715a157017e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:21.971138 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fc0be5b6-f9ef-404b-8e2e-7715a157017e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:22.089534 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0be5b6-f9ef-404b-8e2e-7715a157017e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:56:22.091043 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0be5b6-f9ef-404b-8e2e-7715a157017e service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:56:22.101295 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:22.110278 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 676/1334] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:56:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 15:56:24.180552 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c530e-d5ca-4da7-8a71-5fa2d239be14 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:24.181110 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c530e-d5ca-4da7-8a71-5fa2d239be14 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:24.181110 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c530e-d5ca-4da7-8a71-5fa2d239be14 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:24.182429 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 659/1335] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:56:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:56:24.187533 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9be40-0077-4cea-8d3d-5fece850d714 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:24.187936 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9be40-0077-4cea-8d3d-5fece850d714 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:24.188092 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9be40-0077-4cea-8d3d-5fece850d714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:24.252231 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-16c9be40-0077-4cea-8d3d-5fece850d714 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:56:24.253439 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-16c9be40-0077-4cea-8d3d-5fece850d714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:24.349546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 677/1336] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:56:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:56:24.389811 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c530e-d5ca-4da7-8a71-5fa2d239be14 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:56:24.391341 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2e0c530e-d5ca-4da7-8a71-5fa2d239be14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:24.431171 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:24.493519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c530e-d5ca-4da7-8a71-5fa2d239be14 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:56:24.563984 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5eade-4d90-4af7-86c1-a341d4ce5c2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:56:24.565163 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5eade-4d90-4af7-86c1-a341d4ce5c2f service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:24.565447 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5eade-4d90-4af7-86c1-a341d4ce5c2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:24.565759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5eade-4d90-4af7-86c1-a341d4ce5c2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:24.567420 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2ae5eade-4d90-4af7-86c1-a341d4ce5c2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:24.648954 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:24.674543 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae5eade-4d90-4af7-86c1-a341d4ce5c2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:56:24.676054 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae5eade-4d90-4af7-86c1-a341d4ce5c2f service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:56:24.694302 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 660/1337] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:56:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:56:27.026626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503486a-2714-4933-a46d-82a23178e88f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:27.027192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503486a-2714-4933-a46d-82a23178e88f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:27.027307 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503486a-2714-4933-a46d-82a23178e88f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:27.028612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 678/1338] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:56:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:56:27.033741 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828d866-4626-4287-aa48-2cde20733348 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:27.034176 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828d866-4626-4287-aa48-2cde20733348 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:27.034314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828d866-4626-4287-aa48-2cde20733348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:27.098369 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9828d866-4626-4287-aa48-2cde20733348 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:56:27.099593 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9828d866-4626-4287-aa48-2cde20733348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:27.196892 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 661/1339] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:56:27 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:56:27.233320 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503486a-2714-4933-a46d-82a23178e88f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:56:27.235007 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6503486a-2714-4933-a46d-82a23178e88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:27.310208 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:27.337438 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503486a-2714-4933-a46d-82a23178e88f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:56:27.412083 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c2310-ef5d-4da1-ba0a-3a0900a369bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:56:27.412723 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c2310-ef5d-4da1-ba0a-3a0900a369bd service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:27.413002 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c2310-ef5d-4da1-ba0a-3a0900a369bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:27.413278 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c2310-ef5d-4da1-ba0a-3a0900a369bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:27.414931 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ac2c2310-ef5d-4da1-ba0a-3a0900a369bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:27.522866 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c2310-ef5d-4da1-ba0a-3a0900a369bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:56:27.524650 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c2310-ef5d-4da1-ba0a-3a0900a369bd service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:56:27.542972 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 679/1340] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:56:27 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:56:31.638365 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a7185-f6c8-4a05-8953-e66a87b5eba5 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:31.638897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a7185-f6c8-4a05-8953-e66a87b5eba5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:31.638897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a7185-f6c8-4a05-8953-e66a87b5eba5 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:31.640197 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 662/1341] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:56:31.645599 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfb44d-d5ed-4736-9cd5-0440be6f6ad8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:31.646047 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfb44d-d5ed-4736-9cd5-0440be6f6ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:31.646290 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfb44d-d5ed-4736-9cd5-0440be6f6ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:31.650570 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:31.713873 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-58dfb44d-d5ed-4736-9cd5-0440be6f6ad8 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:56:31.715074 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-58dfb44d-d5ed-4736-9cd5-0440be6f6ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:31.810462 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 680/1342] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:56:31.849227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a7185-f6c8-4a05-8953-e66a87b5eba5 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:56:31.850788 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7c2a7185-f6c8-4a05-8953-e66a87b5eba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:31.899856 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:31.956659 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a7185-f6c8-4a05-8953-e66a87b5eba5 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:56:32.036881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04702f8f-70f3-41e9-8e88-4bc83ea0b64a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:56:32.037531 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04702f8f-70f3-41e9-8e88-4bc83ea0b64a service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:32.037841 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04702f8f-70f3-41e9-8e88-4bc83ea0b64a service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:32.038124 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04702f8f-70f3-41e9-8e88-4bc83ea0b64a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:32.039770 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-04702f8f-70f3-41e9-8e88-4bc83ea0b64a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:32.134272 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:32.150612 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04702f8f-70f3-41e9-8e88-4bc83ea0b64a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:56:32.151991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04702f8f-70f3-41e9-8e88-4bc83ea0b64a service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:56:32.169655 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 663/1343] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:56:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:56:34.798149 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4ac97-76bb-499f-8445-64728ff955af None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:34.799422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4ac97-76bb-499f-8445-64728ff955af None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:34.799422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4ac97-76bb-499f-8445-64728ff955af None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:34.800643 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 681/1344] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:56:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:56:34.806485 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd909ef6-88b5-45a9-b49a-ede9bfc9c70b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:34.807058 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd909ef6-88b5-45a9-b49a-ede9bfc9c70b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:34.807058 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd909ef6-88b5-45a9-b49a-ede9bfc9c70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:34.883641 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-bd909ef6-88b5-45a9-b49a-ede9bfc9c70b None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:56:34.884866 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bd909ef6-88b5-45a9-b49a-ede9bfc9c70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:34.985332 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 664/1345] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:56:34 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:56:35.023752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4ac97-76bb-499f-8445-64728ff955af None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:56:35.025279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-afb4ac97-76bb-499f-8445-64728ff955af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:35.112365 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:35.129054 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4ac97-76bb-499f-8445-64728ff955af None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:56:35.205318 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c204783e-7ddb-404f-9f8d-9165e44b4dfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:56:35.206005 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204783e-7ddb-404f-9f8d-9165e44b4dfb service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:35.206290 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204783e-7ddb-404f-9f8d-9165e44b4dfb service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:35.206566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204783e-7ddb-404f-9f8d-9165e44b4dfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:35.208217 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c204783e-7ddb-404f-9f8d-9165e44b4dfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:35.321369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c204783e-7ddb-404f-9f8d-9165e44b4dfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:56:35.322891 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c204783e-7ddb-404f-9f8d-9165e44b4dfb service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:56:35.341348 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 682/1346] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:56:35 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:56:39.539494 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f10f4-ad7c-446e-86b1-6d278887cf63 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:39.539921 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f10f4-ad7c-446e-86b1-6d278887cf63 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:39.540099 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f10f4-ad7c-446e-86b1-6d278887cf63 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:39.541421 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 665/1347] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:56:39.546739 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c8bc6-0e48-456a-bf6a-6a4bd622539a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:39.547208 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c8bc6-0e48-456a-bf6a-6a4bd622539a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:39.547292 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c8bc6-0e48-456a-bf6a-6a4bd622539a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:39.565420 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:39.620644 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-292c8bc6-0e48-456a-bf6a-6a4bd622539a None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:56:39.621932 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-292c8bc6-0e48-456a-bf6a-6a4bd622539a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:39.730248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 683/1348] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:56:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:56:39.768630 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f10f4-ad7c-446e-86b1-6d278887cf63 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:56:39.770184 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7a2f10f4-ad7c-446e-86b1-6d278887cf63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:39.831400 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:39.882696 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f10f4-ad7c-446e-86b1-6d278887cf63 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:56:39.969457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f391d-94e9-4970-83c5-453133701155 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:56:39.970447 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f391d-94e9-4970-83c5-453133701155 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:39.970731 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f391d-94e9-4970-83c5-453133701155 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:39.971008 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f391d-94e9-4970-83c5-453133701155 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:39.972701 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6f2f391d-94e9-4970-83c5-453133701155 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:40.098487 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:40.103021 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2f391d-94e9-4970-83c5-453133701155 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:56:40.104600 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2f391d-94e9-4970-83c5-453133701155 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:56:40.124568 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 666/1349] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:56:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:56:45.036764 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a09756-74d8-4750-b71e-0b98715a25e3 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:45.037185 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a09756-74d8-4750-b71e-0b98715a25e3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:45.037415 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a09756-74d8-4750-b71e-0b98715a25e3 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:45.039016 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 684/1350] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jul 21 15:56:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:56:45.044379 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8da68-49df-49b4-8b7c-5b6d5b0bac01 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:45.044787 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8da68-49df-49b4-8b7c-5b6d5b0bac01 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:45.044951 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8da68-49df-49b4-8b7c-5b6d5b0bac01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:45.165294 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b3c8da68-49df-49b4-8b7c-5b6d5b0bac01 None None] MFA Rules not processed for user `822d9d3521db4a9688f51c6313c7dd74`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:56:45.166593 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b3c8da68-49df-49b4-8b7c-5b6d5b0bac01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:45.295165 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 667/1351] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jul 21 15:56:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:56:49.555355 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade09119-7219-4ce3-b654-1225a39c89fb None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:49.555891 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade09119-7219-4ce3-b654-1225a39c89fb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:49.555891 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade09119-7219-4ce3-b654-1225a39c89fb None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:49.557177 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 685/1352] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jul 21 15:56:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:56:49.563303 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b2024-8a88-4c02-a1ff-73c2da664725 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:49.564241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b2024-8a88-4c02-a1ff-73c2da664725 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:49.564637 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b2024-8a88-4c02-a1ff-73c2da664725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:49.620738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:49.645493 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-378b2024-8a88-4c02-a1ff-73c2da664725 None None] MFA Rules not processed for user `822d9d3521db4a9688f51c6313c7dd74`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:56:49.646746 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-378b2024-8a88-4c02-a1ff-73c2da664725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:49.728114 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=59 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:49.765587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 668/1353] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jul 21 15:56:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:56:51.368397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbe697-bf5a-4f4e-bdc2-eccac7d8da2f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:51.368953 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbe697-bf5a-4f4e-bdc2-eccac7d8da2f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:51.369064 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbe697-bf5a-4f4e-bdc2-eccac7d8da2f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:51.370744 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 686/1354] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jul 21 15:56:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:56:51.376753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f862e1f-8875-4579-b408-1eac254e1e97 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:51.377504 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f862e1f-8875-4579-b408-1eac254e1e97 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:51.377834 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f862e1f-8875-4579-b408-1eac254e1e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:51.451807 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1f862e1f-8875-4579-b408-1eac254e1e97 None None] MFA Rules not processed for user `822d9d3521db4a9688f51c6313c7dd74`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:56:51.453022 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1f862e1f-8875-4579-b408-1eac254e1e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:51.496500 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:51.568156 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 669/1355] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jul 21 15:56:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 15:56:54.936536 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3e00d-6ec6-45bb-9d2f-a2652c5c2247 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:54.937069 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3e00d-6ec6-45bb-9d2f-a2652c5c2247 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:54.937466 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3e00d-6ec6-45bb-9d2f-a2652c5c2247 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:54.944186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 687/1356] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:56:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:56:54.947159 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4850a-acbd-41ea-9834-7ae0d04c673a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:54.949023 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4850a-acbd-41ea-9834-7ae0d04c673a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:54.949519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4850a-acbd-41ea-9834-7ae0d04c673a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:55.033944 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c7d4850a-acbd-41ea-9834-7ae0d04c673a None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:56:55.035165 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c7d4850a-acbd-41ea-9834-7ae0d04c673a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:55.138486 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 670/1357] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:56:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:56:55.191584 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9838e2-c015-458f-8638-d7a727f54e48 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:55.191873 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9838e2-c015-458f-8638-d7a727f54e48 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:55.192322 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9838e2-c015-458f-8638-d7a727f54e48 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:55.193701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 688/1358] 10.222.0.52 () {54 vars in 962 bytes} [Thu Jul 21 15:56:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:56:55.203936 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18437f-bbdb-49d9-9795-3a50ba882ad3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:55.205238 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18437f-bbdb-49d9-9795-3a50ba882ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:55.205775 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18437f-bbdb-49d9-9795-3a50ba882ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:55.227136 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:55.294530 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9e18437f-bbdb-49d9-9795-3a50ba882ad3 None None] MFA Rules not processed for user `092553a86f8848dba8c7271f5c0b2c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:56:55.295746 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9e18437f-bbdb-49d9-9795-3a50ba882ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:55.452612 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 671/1359] 10.222.0.52 () {58 vars in 1059 bytes} [Thu Jul 21 15:56:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:56:55.460165 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9838e2-c015-458f-8638-d7a727f54e48 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:56:55.461925 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5a9838e2-c015-458f-8638-d7a727f54e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:55.617290 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9838e2-c015-458f-8638-d7a727f54e48 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:56:55.700252 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:55.761138 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d48f1f6-ceeb-4916-9ff2-0083199c014e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:56:55.761138 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48f1f6-ceeb-4916-9ff2-0083199c014e service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:55.761138 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48f1f6-ceeb-4916-9ff2-0083199c014e service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:55.761138 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48f1f6-ceeb-4916-9ff2-0083199c014e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:55.761875 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6d48f1f6-ceeb-4916-9ff2-0083199c014e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:55.860587 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d48f1f6-ceeb-4916-9ff2-0083199c014e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:56:55.862242 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d48f1f6-ceeb-4916-9ff2-0083199c014e service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:56:55.882269 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 689/1360] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:56:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:56:58.184512 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987e047-77c2-49b0-af15-1ce315992a6b None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:58.185035 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987e047-77c2-49b0-af15-1ce315992a6b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:58.185035 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987e047-77c2-49b0-af15-1ce315992a6b None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:58.186586 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 672/1361] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:56:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:56:58.192294 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6e407-c54d-4521-8eeb-2233660412b7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:58.192784 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6e407-c54d-4521-8eeb-2233660412b7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:58.192784 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6e407-c54d-4521-8eeb-2233660412b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:58.267292 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-62d6e407-c54d-4521-8eeb-2233660412b7 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:56:58.268535 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-62d6e407-c54d-4521-8eeb-2233660412b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:58.308150 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:58.373452 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 690/1362] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:56:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:56:58.418885 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6987e047-77c2-49b0-af15-1ce315992a6b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:56:58.421286 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6987e047-77c2-49b0-af15-1ce315992a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:58.497746 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:58.593603 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6987e047-77c2-49b0-af15-1ce315992a6b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:56:58.736809 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86196cea-57d7-4373-9520-0704e241de72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:56:58.737458 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86196cea-57d7-4373-9520-0704e241de72 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:56:58.737773 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86196cea-57d7-4373-9520-0704e241de72 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:56:58.738053 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86196cea-57d7-4373-9520-0704e241de72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:56:58.739709 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-86196cea-57d7-4373-9520-0704e241de72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:56:58.764706 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:56:58.843180 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86196cea-57d7-4373-9520-0704e241de72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:56:58.845024 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86196cea-57d7-4373-9520-0704e241de72 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:56:58.868578 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 673/1363] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:56:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:57:01.249214 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a80e65-4976-445c-8c3c-b8479a9e7c82 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:01.249750 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a80e65-4976-445c-8c3c-b8479a9e7c82 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:01.249994 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a80e65-4976-445c-8c3c-b8479a9e7c82 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:01.252057 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 691/1364] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:57:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:57:01.257616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94c230-1282-45f2-a5a5-347c5230782d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:01.258414 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94c230-1282-45f2-a5a5-347c5230782d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:01.258706 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94c230-1282-45f2-a5a5-347c5230782d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:01.333984 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0a94c230-1282-45f2-a5a5-347c5230782d None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:57:01.333984 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0a94c230-1282-45f2-a5a5-347c5230782d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:01.435226 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:01.438095 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 674/1365] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:57:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:57:01.486797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee1f2e6-3110-4a4c-abf1-66dd39c9ef3e None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:01.487637 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee1f2e6-3110-4a4c-abf1-66dd39c9ef3e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:01.488415 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee1f2e6-3110-4a4c-abf1-66dd39c9ef3e None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:01.490515 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 692/1366] 10.222.0.52 () {54 vars in 962 bytes} [Thu Jul 21 15:57:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:57:01.496556 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2233e-05ce-4944-8129-196dc066c0d7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:01.496984 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2233e-05ce-4944-8129-196dc066c0d7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:01.497445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2233e-05ce-4944-8129-196dc066c0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:01.580842 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8ee2233e-05ce-4944-8129-196dc066c0d7 None None] MFA Rules not processed for user `092553a86f8848dba8c7271f5c0b2c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:57:01.582571 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8ee2233e-05ce-4944-8129-196dc066c0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:01.714959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 675/1367] 10.222.0.52 () {58 vars in 1059 bytes} [Thu Jul 21 15:57:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:57:01.723322 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee1f2e6-3110-4a4c-abf1-66dd39c9ef3e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:57:01.724854 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9ee1f2e6-3110-4a4c-abf1-66dd39c9ef3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:01.889345 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee1f2e6-3110-4a4c-abf1-66dd39c9ef3e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:57:01.911101 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:02.030986 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1fc0c4-fd1f-4cd4-a482-8000c323bdb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:57:02.031563 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1fc0c4-fd1f-4cd4-a482-8000c323bdb9 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:02.031851 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1fc0c4-fd1f-4cd4-a482-8000c323bdb9 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:02.032645 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1fc0c4-fd1f-4cd4-a482-8000c323bdb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:02.034331 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7a1fc0c4-fd1f-4cd4-a482-8000c323bdb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:02.129993 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1fc0c4-fd1f-4cd4-a482-8000c323bdb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:57:02.131622 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1fc0c4-fd1f-4cd4-a482-8000c323bdb9 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:57:02.150443 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 693/1368] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:57:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:57:03.182185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb7cfa-9956-4ecb-bc5b-b2ab70441540 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:03.182714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb7cfa-9956-4ecb-bc5b-b2ab70441540 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:03.182800 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb7cfa-9956-4ecb-bc5b-b2ab70441540 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:03.184290 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 676/1369] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:57:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:57:03.189429 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7678f-4b0f-48c6-b523-df39d598e341 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:03.189867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7678f-4b0f-48c6-b523-df39d598e341 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:03.190040 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7678f-4b0f-48c6-b523-df39d598e341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:03.222361 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:03.266382 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c3e7678f-4b0f-48c6-b523-df39d598e341 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:57:03.268129 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c3e7678f-4b0f-48c6-b523-df39d598e341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:03.372017 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 694/1370] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:57:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:57:03.686548 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479aded-23c0-4a19-8ec0-8bf922b0bccd None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:03.686913 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479aded-23c0-4a19-8ec0-8bf922b0bccd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:03.687266 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479aded-23c0-4a19-8ec0-8bf922b0bccd None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:03.689347 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 677/1371] 10.222.0.52 () {54 vars in 964 bytes} [Thu Jul 21 15:57:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:57:03.695805 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382b34c-9335-45da-beae-1fa750870e7a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:03.696102 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382b34c-9335-45da-beae-1fa750870e7a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:03.696383 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382b34c-9335-45da-beae-1fa750870e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:03.777131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6382b34c-9335-45da-beae-1fa750870e7a None None] MFA Rules not processed for user `87c0598af7714aeba08fffbb6805a49e`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:57:03.778366 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6382b34c-9335-45da-beae-1fa750870e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:03.827448 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:03.861029 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 695/1372] 10.222.0.52 () {58 vars in 1061 bytes} [Thu Jul 21 15:57:03 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:57:03.869438 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8479aded-23c0-4a19-8ec0-8bf922b0bccd None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:57:03.871225 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8479aded-23c0-4a19-8ec0-8bf922b0bccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:03.924522 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:03.987800 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8479aded-23c0-4a19-8ec0-8bf922b0bccd None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:57:04.065430 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4cd2b3-90f4-45a6-98a5-467d6b98c451 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:57:04.066126 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4cd2b3-90f4-45a6-98a5-467d6b98c451 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:04.066407 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4cd2b3-90f4-45a6-98a5-467d6b98c451 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:04.066686 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4cd2b3-90f4-45a6-98a5-467d6b98c451 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:04.068372 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5c4cd2b3-90f4-45a6-98a5-467d6b98c451 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:04.168210 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:04.185219 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4cd2b3-90f4-45a6-98a5-467d6b98c451 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:57:04.186543 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4cd2b3-90f4-45a6-98a5-467d6b98c451 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:57:04.206472 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 678/1373] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:57:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:57:04.310557 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc2e76-1431-4477-ae82-257cc43e4ce4 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:04.311152 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc2e76-1431-4477-ae82-257cc43e4ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:04.311152 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc2e76-1431-4477-ae82-257cc43e4ce4 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:04.317752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 696/1374] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:57:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:57:04.319232 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b2630-a0d1-4819-a3e3-03380e6896f6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:04.319232 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b2630-a0d1-4819-a3e3-03380e6896f6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:04.319232 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b2630-a0d1-4819-a3e3-03380e6896f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:04.390809 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-403b2630-a0d1-4819-a3e3-03380e6896f6 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:57:04.392260 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-403b2630-a0d1-4819-a3e3-03380e6896f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:04.521257 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 679/1375] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:57:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:57:04.565899 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdc2e76-1431-4477-ae82-257cc43e4ce4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:57:04.567605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-acdc2e76-1431-4477-ae82-257cc43e4ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:04.715467 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdc2e76-1431-4477-ae82-257cc43e4ce4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:57:04.765849 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:04.827286 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5af685-4f18-4590-b524-4e030f92b0ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:57:04.827933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5af685-4f18-4590-b524-4e030f92b0ea service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:04.828217 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5af685-4f18-4590-b524-4e030f92b0ea service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:04.828491 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5af685-4f18-4590-b524-4e030f92b0ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:04.830727 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2d5af685-4f18-4590-b524-4e030f92b0ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:04.926304 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5af685-4f18-4590-b524-4e030f92b0ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:57:04.927931 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5af685-4f18-4590-b524-4e030f92b0ea service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:57:04.946794 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 697/1376] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:57:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:57:07.055318 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2842945-6825-4c92-b65a-48ed141c56ae None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:07.055838 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2842945-6825-4c92-b65a-48ed141c56ae None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:07.055960 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2842945-6825-4c92-b65a-48ed141c56ae None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:07.057251 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 680/1377] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:57:07.062483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbac76e-666d-4427-966f-4682e763f7f8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:07.062909 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbac76e-666d-4427-966f-4682e763f7f8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:07.063168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbac76e-666d-4427-966f-4682e763f7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:07.130423 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0cbac76e-666d-4427-966f-4682e763f7f8 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:57:07.131635 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0cbac76e-666d-4427-966f-4682e763f7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:07.138206 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:07.242535 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 698/1378] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:57:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:57:07.289312 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2842945-6825-4c92-b65a-48ed141c56ae None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:57:07.290902 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c2842945-6825-4c92-b65a-48ed141c56ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:07.367169 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:07.452620 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2842945-6825-4c92-b65a-48ed141c56ae None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:57:07.568087 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd2a74f-1baf-4295-9327-a457edb6abca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:57:07.568741 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2a74f-1baf-4295-9327-a457edb6abca service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:07.569037 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2a74f-1baf-4295-9327-a457edb6abca service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:07.569316 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2a74f-1baf-4295-9327-a457edb6abca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:07.570999 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4cd2a74f-1baf-4295-9327-a457edb6abca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:07.589426 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:07.666100 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd2a74f-1baf-4295-9327-a457edb6abca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:57:07.668249 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd2a74f-1baf-4295-9327-a457edb6abca service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:57:07.692477 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 681/1379] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:57:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:57:09.809241 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0abb2-391e-4132-9233-821612e53be9 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:09.809807 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0abb2-391e-4132-9233-821612e53be9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:09.809807 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0abb2-391e-4132-9233-821612e53be9 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:09.811337 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 699/1380] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:57:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:57:09.816522 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee84596-147b-4765-9bed-de0997e43b6f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:09.817045 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee84596-147b-4765-9bed-de0997e43b6f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:09.817172 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee84596-147b-4765-9bed-de0997e43b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:09.895116 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0ee84596-147b-4765-9bed-de0997e43b6f None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:57:09.896476 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0ee84596-147b-4765-9bed-de0997e43b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:09.994199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:10.009166 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 682/1381] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:57:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:57:10.045416 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db0abb2-391e-4132-9233-821612e53be9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:57:10.046953 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7db0abb2-391e-4132-9233-821612e53be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:10.196874 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:10.209679 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db0abb2-391e-4132-9233-821612e53be9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:57:10.333038 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f617b5-e795-48ee-8764-b1651ff5065e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:57:10.333623 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f617b5-e795-48ee-8764-b1651ff5065e service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:10.333912 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f617b5-e795-48ee-8764-b1651ff5065e service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:10.334188 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f617b5-e795-48ee-8764-b1651ff5065e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:10.335858 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-15f617b5-e795-48ee-8764-b1651ff5065e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:10.428726 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f617b5-e795-48ee-8764-b1651ff5065e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:57:10.430583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f617b5-e795-48ee-8764-b1651ff5065e service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:57:10.442140 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:10.462652 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 700/1382] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:57:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:57:12.537302 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d92e6e-7321-424f-959e-c8e332739c2b None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:12.537867 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d92e6e-7321-424f-959e-c8e332739c2b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:12.537867 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d92e6e-7321-424f-959e-c8e332739c2b None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:12.539181 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 683/1383] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:57:12.544533 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cfbfcd-6e6e-4173-aa0e-790d1da3dfd9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:12.544970 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cfbfcd-6e6e-4173-aa0e-790d1da3dfd9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:12.545084 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cfbfcd-6e6e-4173-aa0e-790d1da3dfd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:12.611853 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-48cfbfcd-6e6e-4173-aa0e-790d1da3dfd9 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:57:12.613311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-48cfbfcd-6e6e-4173-aa0e-790d1da3dfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:12.719217 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 701/1384] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:57:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:57:12.770810 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d92e6e-7321-424f-959e-c8e332739c2b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:57:12.772358 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f5d92e6e-7321-424f-959e-c8e332739c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:12.920901 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d92e6e-7321-424f-959e-c8e332739c2b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:57:12.992944 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:13.047494 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd934d9e-ee5f-4625-85b1-d87e8db76648 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:57:13.048152 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd934d9e-ee5f-4625-85b1-d87e8db76648 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:13.048442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd934d9e-ee5f-4625-85b1-d87e8db76648 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:13.048726 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd934d9e-ee5f-4625-85b1-d87e8db76648 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:13.050515 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cd934d9e-ee5f-4625-85b1-d87e8db76648 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:13.151885 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd934d9e-ee5f-4625-85b1-d87e8db76648 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:57:13.154003 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd934d9e-ee5f-4625-85b1-d87e8db76648 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:57:13.178340 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 684/1385] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:57:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:57:15.820453 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1d420-5167-4199-8471-91c78e06ff87 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:15.821148 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1d420-5167-4199-8471-91c78e06ff87 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:15.821148 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1d420-5167-4199-8471-91c78e06ff87 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:15.822429 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 702/1386] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:57:15.827606 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb160d5e-cf12-4db7-a883-502887c95a5b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:15.828034 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb160d5e-cf12-4db7-a883-502887c95a5b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:15.828170 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb160d5e-cf12-4db7-a883-502887c95a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:15.894019 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-eb160d5e-cf12-4db7-a883-502887c95a5b None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:57:15.895244 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-eb160d5e-cf12-4db7-a883-502887c95a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:15.927859 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:16.005458 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 685/1387] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:57:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:57:16.050531 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d1d420-5167-4199-8471-91c78e06ff87 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:57:16.052585 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-87d1d420-5167-4199-8471-91c78e06ff87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:16.133357 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:16.212546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d1d420-5167-4199-8471-91c78e06ff87 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:57:16.324908 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4998e2-3470-425f-a9c3-2ca2892f157b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:57:16.325475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4998e2-3470-425f-a9c3-2ca2892f157b service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:16.325809 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4998e2-3470-425f-a9c3-2ca2892f157b service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:16.326062 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4998e2-3470-425f-a9c3-2ca2892f157b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:16.327702 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7e4998e2-3470-425f-a9c3-2ca2892f157b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:16.365593 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:16.427231 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4998e2-3470-425f-a9c3-2ca2892f157b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:57:16.428846 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4998e2-3470-425f-a9c3-2ca2892f157b service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:57:16.447339 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 703/1388] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:57:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:57:18.514932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad331c-222a-41de-8949-05f8ec532b6a None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:18.515439 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad331c-222a-41de-8949-05f8ec532b6a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:18.515439 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad331c-222a-41de-8949-05f8ec532b6a None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:18.516814 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 686/1389] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:57:18.522193 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3394d-2989-4c18-945a-3dee98240746 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:18.522583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3394d-2989-4c18-945a-3dee98240746 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:18.522775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3394d-2989-4c18-945a-3dee98240746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:18.586130 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e2f3394d-2989-4c18-945a-3dee98240746 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:57:18.587368 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e2f3394d-2989-4c18-945a-3dee98240746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:18.683468 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 704/1390] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:57:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:57:18.727905 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad331c-222a-41de-8949-05f8ec532b6a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:57:18.729536 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-07ad331c-222a-41de-8949-05f8ec532b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:18.879420 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad331c-222a-41de-8949-05f8ec532b6a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:57:18.897133 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:18.995020 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e9400-291d-4ddf-9476-a3a14b40ba29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:57:18.995666 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e9400-291d-4ddf-9476-a3a14b40ba29 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:18.995950 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e9400-291d-4ddf-9476-a3a14b40ba29 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:18.996228 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e9400-291d-4ddf-9476-a3a14b40ba29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:18.997884 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ec1e9400-291d-4ddf-9476-a3a14b40ba29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:19.087561 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1e9400-291d-4ddf-9476-a3a14b40ba29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:57:19.089182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1e9400-291d-4ddf-9476-a3a14b40ba29 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:57:19.107508 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 687/1391] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:57:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:57:19.880179 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14945838-636a-4777-994e-9e6c3d3bb3c1 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:19.880727 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14945838-636a-4777-994e-9e6c3d3bb3c1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:19.880727 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14945838-636a-4777-994e-9e6c3d3bb3c1 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:19.882121 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 705/1392] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:57:19.887566 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e608bb2-7109-46af-b9c9-420d7cd80c5f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:19.888018 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e608bb2-7109-46af-b9c9-420d7cd80c5f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:19.888251 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e608bb2-7109-46af-b9c9-420d7cd80c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:19.913172 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:19.953164 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1e608bb2-7109-46af-b9c9-420d7cd80c5f None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:57:19.955004 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1e608bb2-7109-46af-b9c9-420d7cd80c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:20.050947 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 688/1393] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:57:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 15:57:20.106623 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc8f81-ee3d-49b1-a4ff-0c6b314bb20d None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:20.106913 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc8f81-ee3d-49b1-a4ff-0c6b314bb20d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:20.107187 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc8f81-ee3d-49b1-a4ff-0c6b314bb20d None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:20.108550 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 706/1394] 10.222.0.52 () {54 vars in 966 bytes} [Thu Jul 21 15:57:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:57:20.113608 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0aeaca-c555-4de0-9a88-14058e03f461 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:20.113918 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0aeaca-c555-4de0-9a88-14058e03f461 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:20.114204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0aeaca-c555-4de0-9a88-14058e03f461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:20.194934 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-fb0aeaca-c555-4de0-9a88-14058e03f461 None None] MFA Rules not processed for user `d73a7d5f37e144768d7cc53f13cd5157`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:57:20.196068 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fb0aeaca-c555-4de0-9a88-14058e03f461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:20.243950 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:20.281654 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 689/1395] 10.222.0.52 () {58 vars in 1063 bytes} [Thu Jul 21 15:57:20 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:57:20.289182 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dc8f81-ee3d-49b1-a4ff-0c6b314bb20d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:57:20.290856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-69dc8f81-ee3d-49b1-a4ff-0c6b314bb20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:20.304560 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:20.397441 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dc8f81-ee3d-49b1-a4ff-0c6b314bb20d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:57:20.469419 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71c14a-7a94-450a-b80e-78394084c7f4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73a7d5f37e144768d7cc53f13cd5157', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:57:20.470110 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71c14a-7a94-450a-b80e-78394084c7f4 service glance] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:20.470401 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71c14a-7a94-450a-b80e-78394084c7f4 service glance] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:20.470941 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71c14a-7a94-450a-b80e-78394084c7f4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:20.472602 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8e71c14a-7a94-450a-b80e-78394084c7f4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:20.539361 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:20.582155 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e71c14a-7a94-450a-b80e-78394084c7f4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:57:20.583601 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e71c14a-7a94-450a-b80e-78394084c7f4 service glance] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:57:20.601589 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 707/1396] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:57:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:57:21.156282 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c3708-fad3-40fa-b2a8-6ead3937324f None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:21.156909 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c3708-fad3-40fa-b2a8-6ead3937324f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:21.156909 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c3708-fad3-40fa-b2a8-6ead3937324f None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:21.158867 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 690/1397] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:57:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:57:21.164985 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0fc84-f34b-4bbe-97d5-13d20857c1bd None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:21.165527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0fc84-f34b-4bbe-97d5-13d20857c1bd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:21.165687 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0fc84-f34b-4bbe-97d5-13d20857c1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:21.243469 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-10f0fc84-f34b-4bbe-97d5-13d20857c1bd None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:57:21.244709 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-10f0fc84-f34b-4bbe-97d5-13d20857c1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:21.331542 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2258d9b-ba2f-4de0-b177-4385b0c87fd6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:21.331945 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2258d9b-ba2f-4de0-b177-4385b0c87fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:21.332296 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2258d9b-ba2f-4de0-b177-4385b0c87fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:21.353487 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 708/1398] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:57:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:57:21.398387 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0fc84-f34b-4bbe-97d5-13d20857c1bd None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:57:21.399742 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-10f0fc84-f34b-4bbe-97d5-13d20857c1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:21.400772 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b2258d9b-ba2f-4de0-b177-4385b0c87fd6 None None] MFA Rules not processed for user `6edbad98ed0f4823b9955f9f0e270149`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:57:21.402012 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b2258d9b-ba2f-4de0-b177-4385b0c87fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:21.427443 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:21.495947 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 691/1399] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 21 15:57:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 15:57:21.519359 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0fc84-f34b-4bbe-97d5-13d20857c1bd None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:57:21.539481 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b5912-aeed-432c-8fa3-de649b7194c5 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:21.539799 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b5912-aeed-432c-8fa3-de649b7194c5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:21.540117 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b5912-aeed-432c-8fa3-de649b7194c5 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:21.541694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 692/1400] 10.222.0.52 () {54 vars in 943 bytes} [Thu Jul 21 15:57:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 15:57:21.549270 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a15af1-ecaa-4d6e-8f95-09e78bccac00 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:21.549564 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a15af1-ecaa-4d6e-8f95-09e78bccac00 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:21.549866 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a15af1-ecaa-4d6e-8f95-09e78bccac00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:21.626561 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:21.636535 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba16ec-8ac5-4188-82dc-7f136f1667a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:57:21.636946 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-48a15af1-ecaa-4d6e-8f95-09e78bccac00 None None] MFA Rules not processed for user `f5bfae2cb30843b1b1a0343efef38638`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:57:21.637189 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba16ec-8ac5-4188-82dc-7f136f1667a3 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:21.637729 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba16ec-8ac5-4188-82dc-7f136f1667a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:21.638071 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba16ec-8ac5-4188-82dc-7f136f1667a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:21.638403 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-48a15af1-ecaa-4d6e-8f95-09e78bccac00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:21.640332 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-60ba16ec-8ac5-4188-82dc-7f136f1667a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:21.662419 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:21.741665 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ba16ec-8ac5-4188-82dc-7f136f1667a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:57:21.743307 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ba16ec-8ac5-4188-82dc-7f136f1667a3 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:57:21.758402 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 693/1401] 10.222.0.52 () {58 vars in 1040 bytes} [Thu Jul 21 15:57:21 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:57:21.762073 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 709/1402] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:57:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 15:57:21.767637 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a15af1-ecaa-4d6e-8f95-09e78bccac00 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:57:21.777609 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a15af1-ecaa-4d6e-8f95-09e78bccac00 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:57:21.898473 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c0bae-e1f7-4166-8701-099d15c3cbf5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5bfae2cb30843b1b1a0343efef38638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:57:21.899026 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c0bae-e1f7-4166-8701-099d15c3cbf5 service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:21.899313 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c0bae-e1f7-4166-8701-099d15c3cbf5 service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:21.899596 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c0bae-e1f7-4166-8701-099d15c3cbf5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:21.901373 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4b7c0bae-e1f7-4166-8701-099d15c3cbf5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:21.904489 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:21.997000 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7c0bae-e1f7-4166-8701-099d15c3cbf5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:57:21.998668 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7c0bae-e1f7-4166-8701-099d15c3cbf5 service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:57:22.007385 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 694/1403] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 15:57:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 15:57:23.866127 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e5446-131b-4ecb-962b-01a3f5b34382 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:23.866719 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e5446-131b-4ecb-962b-01a3f5b34382 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:23.866719 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e5446-131b-4ecb-962b-01a3f5b34382 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:23.868323 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 710/1404] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:57:23.873706 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a0dc0-4f74-4d28-be50-04cc270393db None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:23.875270 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a0dc0-4f74-4d28-be50-04cc270393db None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:23.876072 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a0dc0-4f74-4d28-be50-04cc270393db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:23.950358 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7a0a0dc0-4f74-4d28-be50-04cc270393db None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:57:23.953638 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7a0a0dc0-4f74-4d28-be50-04cc270393db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:24.089943 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:24.099369 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 695/1405] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:57:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:57:24.135926 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e5446-131b-4ecb-962b-01a3f5b34382 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:57:24.137464 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0f7e5446-131b-4ecb-962b-01a3f5b34382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:24.325987 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:24.332808 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e5446-131b-4ecb-962b-01a3f5b34382 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:57:24.459903 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615fb56f-d5e1-4784-89b4-466f30656598 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:57:24.460550 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615fb56f-d5e1-4784-89b4-466f30656598 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:24.460832 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615fb56f-d5e1-4784-89b4-466f30656598 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:24.461107 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615fb56f-d5e1-4784-89b4-466f30656598 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:24.462798 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-615fb56f-d5e1-4784-89b4-466f30656598 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:24.562519 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615fb56f-d5e1-4784-89b4-466f30656598 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:57:24.564637 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615fb56f-d5e1-4784-89b4-466f30656598 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:57:24.584550 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 711/1406] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:57:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:57:26.661019 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046af30-a24c-4fb6-a6df-acca07f5e383 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:26.661540 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046af30-a24c-4fb6-a6df-acca07f5e383 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:26.661540 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046af30-a24c-4fb6-a6df-acca07f5e383 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:26.662907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 696/1407] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:57:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:57:26.668309 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9cada-7321-41dc-b507-d8c30f5bf043 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:26.668711 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9cada-7321-41dc-b507-d8c30f5bf043 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:26.668869 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9cada-7321-41dc-b507-d8c30f5bf043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:26.672159 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:26.741788 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-25c9cada-7321-41dc-b507-d8c30f5bf043 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:57:26.743042 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-25c9cada-7321-41dc-b507-d8c30f5bf043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:26.853607 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 712/1408] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:57:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:57:26.899568 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5046af30-a24c-4fb6-a6df-acca07f5e383 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:57:26.901131 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5046af30-a24c-4fb6-a6df-acca07f5e383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:27.075370 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5046af30-a24c-4fb6-a6df-acca07f5e383 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:57:27.156413 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:27.207811 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae832c44-c380-4b41-a130-5bc46f1a19f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:57:27.208575 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae832c44-c380-4b41-a130-5bc46f1a19f1 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:27.208939 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae832c44-c380-4b41-a130-5bc46f1a19f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:27.209289 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae832c44-c380-4b41-a130-5bc46f1a19f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:27.211054 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ae832c44-c380-4b41-a130-5bc46f1a19f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:27.319324 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae832c44-c380-4b41-a130-5bc46f1a19f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:57:27.320955 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae832c44-c380-4b41-a130-5bc46f1a19f1 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:57:27.341107 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 697/1409] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:57:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:57:36.850327 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=64 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:37.338597 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=64 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:44.620253 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15181a-7e73-411f-9e8f-52f099e4d230 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:44.620788 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15181a-7e73-411f-9e8f-52f099e4d230 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:44.620788 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15181a-7e73-411f-9e8f-52f099e4d230 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:44.622127 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 713/1410] 10.222.0.52 () {54 vars in 918 bytes} [Thu Jul 21 15:57:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:57:44.627194 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0729cf-f076-49d8-b598-96d5ff03ddbc None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:44.627591 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0729cf-f076-49d8-b598-96d5ff03ddbc None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:44.627752 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0729cf-f076-49d8-b598-96d5ff03ddbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:44.702252 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1c0729cf-f076-49d8-b598-96d5ff03ddbc None None] MFA Rules not processed for user `822d9d3521db4a9688f51c6313c7dd74`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:57:44.703982 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1c0729cf-f076-49d8-b598-96d5ff03ddbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:44.809360 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 698/1411] 10.222.0.52 () {58 vars in 1015 bytes} [Thu Jul 21 15:57:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 15:57:44.824158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508e5a6-38ae-4e60-8ea2-4cf49d49b911 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:44.831759 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508e5a6-38ae-4e60-8ea2-4cf49d49b911 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:44.831759 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508e5a6-38ae-4e60-8ea2-4cf49d49b911 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:44.831759 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 714/1412] 10.222.0.52 () {54 vars in 918 bytes} [Thu Jul 21 15:57:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:57:44.833453 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65c924-588a-40f2-ab40-e6a97e4da528 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:44.833453 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65c924-588a-40f2-ab40-e6a97e4da528 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:44.833453 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65c924-588a-40f2-ab40-e6a97e4da528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:44.904001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:44.906829 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-df65c924-588a-40f2-ab40-e6a97e4da528 None None] MFA Rules not processed for user `092553a86f8848dba8c7271f5c0b2c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:57:44.908594 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-df65c924-588a-40f2-ab40-e6a97e4da528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:45.026074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 699/1413] 10.222.0.52 () {58 vars in 1015 bytes} [Thu Jul 21 15:57:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:57:47.536488 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a123c-b39d-43f6-9b8c-a456612d2f42 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:47.537055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a123c-b39d-43f6-9b8c-a456612d2f42 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:47.537055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a123c-b39d-43f6-9b8c-a456612d2f42 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:47.538454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 715/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 15:57:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:57:47.543680 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e2d57-fd4f-4465-9b12-00719fdd4295 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:47.544136 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e2d57-fd4f-4465-9b12-00719fdd4295 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:47.544267 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e2d57-fd4f-4465-9b12-00719fdd4295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:47.609305 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-384e2d57-fd4f-4465-9b12-00719fdd4295 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:57:47.610561 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-384e2d57-fd4f-4465-9b12-00719fdd4295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:47.664483 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:47.706138 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 700/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 15:57:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:57:47.748021 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a123c-b39d-43f6-9b8c-a456612d2f42 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:57:47.749661 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-506a123c-b39d-43f6-9b8c-a456612d2f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:47.895255 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a123c-b39d-43f6-9b8c-a456612d2f42 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:57:47.967184 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:48.007741 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2078f0cf-0dcc-4e6b-ab01-7365554a589f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:57:48.008385 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078f0cf-0dcc-4e6b-ab01-7365554a589f service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:48.008668 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078f0cf-0dcc-4e6b-ab01-7365554a589f service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:48.008945 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078f0cf-0dcc-4e6b-ab01-7365554a589f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:48.010620 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2078f0cf-0dcc-4e6b-ab01-7365554a589f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:48.098578 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2078f0cf-0dcc-4e6b-ab01-7365554a589f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:57:48.100184 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2078f0cf-0dcc-4e6b-ab01-7365554a589f service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:57:48.120338 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 716/1416] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:57:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:57:49.022492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77db8d7-c8f5-43a7-83c0-f55f5bb4c457 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:49.023060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77db8d7-c8f5-43a7-83c0-f55f5bb4c457 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:49.023178 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77db8d7-c8f5-43a7-83c0-f55f5bb4c457 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:49.024553 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 701/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 15:57:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:57:49.029408 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05bebff-88db-40a6-b0ac-c385c06a71ac None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:49.029858 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05bebff-88db-40a6-b0ac-c385c06a71ac None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:49.029991 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05bebff-88db-40a6-b0ac-c385c06a71ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:49.104229 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d05bebff-88db-40a6-b0ac-c385c06a71ac None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:57:49.105416 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d05bebff-88db-40a6-b0ac-c385c06a71ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:49.132296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:49.210048 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 717/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 15:57:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:57:49.360525 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77db8d7-c8f5-43a7-83c0-f55f5bb4c457 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:57:49.362073 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a77db8d7-c8f5-43a7-83c0-f55f5bb4c457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:49.502734 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77db8d7-c8f5-43a7-83c0-f55f5bb4c457 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:57:49.541912 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:49.614503 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c161b429-5c0b-41dd-ba91-c378d45378cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:57:49.615082 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161b429-5c0b-41dd-ba91-c378d45378cd service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:49.615370 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161b429-5c0b-41dd-ba91-c378d45378cd service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:49.615660 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161b429-5c0b-41dd-ba91-c378d45378cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:49.617386 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c161b429-5c0b-41dd-ba91-c378d45378cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:49.705236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c161b429-5c0b-41dd-ba91-c378d45378cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:57:49.706943 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c161b429-5c0b-41dd-ba91-c378d45378cd service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:57:49.724840 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 702/1419] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:57:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:57:50.596759 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85755657-9dd3-426a-bd01-e2671b30047b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:50.597342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85755657-9dd3-426a-bd01-e2671b30047b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:50.597342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85755657-9dd3-426a-bd01-e2671b30047b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:50.598644 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 718/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 15:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:57:50.603598 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342d487-1cfd-4360-9e35-030116449009 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:50.604028 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342d487-1cfd-4360-9e35-030116449009 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:50.604164 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342d487-1cfd-4360-9e35-030116449009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:50.657586 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:50.679319 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6342d487-1cfd-4360-9e35-030116449009 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:57:50.680558 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6342d487-1cfd-4360-9e35-030116449009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:50.777142 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 703/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 15:57:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:57:50.930300 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85755657-9dd3-426a-bd01-e2671b30047b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:57:50.931746 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-85755657-9dd3-426a-bd01-e2671b30047b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:51.076855 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85755657-9dd3-426a-bd01-e2671b30047b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:57:51.088844 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:51.191217 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f172e45-8972-4389-b261-1027cfd30638 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:57:51.191859 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f172e45-8972-4389-b261-1027cfd30638 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:51.192142 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f172e45-8972-4389-b261-1027cfd30638 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:51.192418 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f172e45-8972-4389-b261-1027cfd30638 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:51.194220 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1f172e45-8972-4389-b261-1027cfd30638 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:51.286865 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f172e45-8972-4389-b261-1027cfd30638 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:57:51.288474 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f172e45-8972-4389-b261-1027cfd30638 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:57:51.306977 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 719/1422] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:57:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:57:52.159027 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b889e-c8cc-466a-9026-9c1cd02e7422 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:52.159557 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b889e-c8cc-466a-9026-9c1cd02e7422 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:52.159675 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b889e-c8cc-466a-9026-9c1cd02e7422 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:52.160979 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 704/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 15:57:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:57:52.166015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825e4ef-65f1-42db-bca0-994ad4a9d8bb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:52.166448 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825e4ef-65f1-42db-bca0-994ad4a9d8bb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:52.166563 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825e4ef-65f1-42db-bca0-994ad4a9d8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:52.185110 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:52.244825 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f825e4ef-65f1-42db-bca0-994ad4a9d8bb None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:57:52.247026 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f825e4ef-65f1-42db-bca0-994ad4a9d8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:52.346794 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 720/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 15:57:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:57:52.495405 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b889e-c8cc-466a-9026-9c1cd02e7422 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:57:52.496873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-eb0b889e-c8cc-466a-9026-9c1cd02e7422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:52.608808 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:52.651012 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b889e-c8cc-466a-9026-9c1cd02e7422 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:57:52.761972 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a0afc-70ab-4f48-a485-9e9f033069f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:57:52.762544 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a0afc-70ab-4f48-a485-9e9f033069f3 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:52.762831 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a0afc-70ab-4f48-a485-9e9f033069f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:52.763107 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a0afc-70ab-4f48-a485-9e9f033069f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:52.764765 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bb8a0afc-70ab-4f48-a485-9e9f033069f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:52.839174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:52.853128 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8a0afc-70ab-4f48-a485-9e9f033069f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:57:52.854759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8a0afc-70ab-4f48-a485-9e9f033069f3 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:57:52.873072 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 705/1425] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:57:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:57:53.742239 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4efafe9-e833-4c40-a8d4-9731b2700088 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:53.742781 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4efafe9-e833-4c40-a8d4-9731b2700088 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:53.742781 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4efafe9-e833-4c40-a8d4-9731b2700088 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:53.744160 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 721/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 15:57:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:57:53.749135 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284cd9cd-8fd0-4059-9dfa-ed7bdb8c9ddb None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:53.749537 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284cd9cd-8fd0-4059-9dfa-ed7bdb8c9ddb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:53.749731 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284cd9cd-8fd0-4059-9dfa-ed7bdb8c9ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:53.824087 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-284cd9cd-8fd0-4059-9dfa-ed7bdb8c9ddb None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:57:53.825300 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-284cd9cd-8fd0-4059-9dfa-ed7bdb8c9ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:53.920447 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 706/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 15:57:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:57:54.069035 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4efafe9-e833-4c40-a8d4-9731b2700088 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:57:54.070559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f4efafe9-e833-4c40-a8d4-9731b2700088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:54.153058 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:54.214650 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4efafe9-e833-4c40-a8d4-9731b2700088 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:57:54.334771 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd40beea-7249-4a3c-984d-66418ae5c303 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:57:54.335335 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40beea-7249-4a3c-984d-66418ae5c303 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:54.335615 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40beea-7249-4a3c-984d-66418ae5c303 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:54.335889 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40beea-7249-4a3c-984d-66418ae5c303 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:54.337534 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fd40beea-7249-4a3c-984d-66418ae5c303 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:54.385377 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:54.427323 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd40beea-7249-4a3c-984d-66418ae5c303 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:57:54.428962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd40beea-7249-4a3c-984d-66418ae5c303 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:57:54.447046 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 722/1428] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:57:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:57:55.245327 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c40c11-7bb8-4a93-aecc-5cb57ef9a459 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:55.245818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c40c11-7bb8-4a93-aecc-5cb57ef9a459 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:55.245933 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c40c11-7bb8-4a93-aecc-5cb57ef9a459 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:55.247302 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 707/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 15:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:57:55.253018 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ff0ae-da1f-487e-afeb-0ee76725f684 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:55.253456 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ff0ae-da1f-487e-afeb-0ee76725f684 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:55.253560 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ff0ae-da1f-487e-afeb-0ee76725f684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:55.329761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-944ff0ae-da1f-487e-afeb-0ee76725f684 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:57:55.331001 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-944ff0ae-da1f-487e-afeb-0ee76725f684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:55.426751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 723/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 15:57:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:57:55.575722 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c40c11-7bb8-4a93-aecc-5cb57ef9a459 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:57:55.577184 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-22c40c11-7bb8-4a93-aecc-5cb57ef9a459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:55.626311 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:55.720352 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c40c11-7bb8-4a93-aecc-5cb57ef9a459 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:57:55.831909 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d39484-5bc4-4f2a-9dfc-a3977461d3fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:57:55.832485 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d39484-5bc4-4f2a-9dfc-a3977461d3fb service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:55.832773 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d39484-5bc4-4f2a-9dfc-a3977461d3fb service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:55.833055 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d39484-5bc4-4f2a-9dfc-a3977461d3fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:55.834749 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c7d39484-5bc4-4f2a-9dfc-a3977461d3fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:55.841593 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:55.923738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d39484-5bc4-4f2a-9dfc-a3977461d3fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:57:55.925344 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d39484-5bc4-4f2a-9dfc-a3977461d3fb service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:57:55.943638 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 708/1431] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:57:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:57:56.748247 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461049e8-1f1b-42e8-b900-c1dd2475f827 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:56.748795 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461049e8-1f1b-42e8-b900-c1dd2475f827 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:56.748795 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461049e8-1f1b-42e8-b900-c1dd2475f827 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:56.750310 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 724/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 15:57:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:57:56.755289 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece072dc-03cd-4d9d-ad1f-b0c5a376e09a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:56.755736 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece072dc-03cd-4d9d-ad1f-b0c5a376e09a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:56.755886 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece072dc-03cd-4d9d-ad1f-b0c5a376e09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:56.830415 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ece072dc-03cd-4d9d-ad1f-b0c5a376e09a None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:57:56.831644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ece072dc-03cd-4d9d-ad1f-b0c5a376e09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:56.899267 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:56.931872 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 709/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 15:57:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:57:57.080434 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461049e8-1f1b-42e8-b900-c1dd2475f827 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:57:57.081907 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-461049e8-1f1b-42e8-b900-c1dd2475f827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:57.100550 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:57.231958 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461049e8-1f1b-42e8-b900-c1dd2475f827 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:57:57.319190 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:57.345867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb8f19-55e9-4f16-bd5a-9fcfc7986226 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:57:57.346745 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb8f19-55e9-4f16-bd5a-9fcfc7986226 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:57.347032 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb8f19-55e9-4f16-bd5a-9fcfc7986226 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:57.347354 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb8f19-55e9-4f16-bd5a-9fcfc7986226 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:57.349071 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-47cb8f19-55e9-4f16-bd5a-9fcfc7986226 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:57.436332 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb8f19-55e9-4f16-bd5a-9fcfc7986226 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:57:57.438095 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb8f19-55e9-4f16-bd5a-9fcfc7986226 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:57:57.456336 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 725/1434] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:57:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:57:58.267255 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912515e0-80df-49ad-8d9e-f798a9e372a6 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:58.267768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912515e0-80df-49ad-8d9e-f798a9e372a6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:58.267900 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912515e0-80df-49ad-8d9e-f798a9e372a6 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:58.269199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 710/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 15:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:57:58.274262 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29c5ec-1277-4928-be04-4bb01060a407 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:58.274652 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29c5ec-1277-4928-be04-4bb01060a407 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:58.274833 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29c5ec-1277-4928-be04-4bb01060a407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:58.349500 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ed29c5ec-1277-4928-be04-4bb01060a407 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:57:58.350722 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ed29c5ec-1277-4928-be04-4bb01060a407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:58.391261 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:58.446205 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 726/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 15:57:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:57:58.594066 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912515e0-80df-49ad-8d9e-f798a9e372a6 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:57:58.595512 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-912515e0-80df-49ad-8d9e-f798a9e372a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:58.738453 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912515e0-80df-49ad-8d9e-f798a9e372a6 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:57:58.791749 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:58.850817 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362473a-cb2b-4637-a9e3-ad2f089e5904 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:57:58.851385 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362473a-cb2b-4637-a9e3-ad2f089e5904 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:58.851674 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362473a-cb2b-4637-a9e3-ad2f089e5904 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:58.851953 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362473a-cb2b-4637-a9e3-ad2f089e5904 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:58.853623 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3362473a-cb2b-4637-a9e3-ad2f089e5904 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:58.941840 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362473a-cb2b-4637-a9e3-ad2f089e5904 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:57:58.943488 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362473a-cb2b-4637-a9e3-ad2f089e5904 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:57:58.961754 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 711/1437] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:57:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:57:59.775692 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d96096-5f83-47d4-b3ff-0225fe51f16d None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:59.776258 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d96096-5f83-47d4-b3ff-0225fe51f16d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:59.776258 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d96096-5f83-47d4-b3ff-0225fe51f16d None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:59.777524 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 727/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 15:57:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:57:59.782897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df003f-af66-4a65-97a4-2cc69fd82a4a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:57:59.783345 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df003f-af66-4a65-97a4-2cc69fd82a4a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:57:59.783484 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df003f-af66-4a65-97a4-2cc69fd82a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:57:59.857687 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a4df003f-af66-4a65-97a4-2cc69fd82a4a None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:57:59.858898 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a4df003f-af66-4a65-97a4-2cc69fd82a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:57:59.861667 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:57:59.972764 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 712/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 15:57:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:58:00.121925 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d96096-5f83-47d4-b3ff-0225fe51f16d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:58:00.123362 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-94d96096-5f83-47d4-b3ff-0225fe51f16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:00.271248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d96096-5f83-47d4-b3ff-0225fe51f16d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:58:00.298055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:58:00.383710 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9383f82-2c0d-46fc-ac1d-2faeb111acaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:58:00.384296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9383f82-2c0d-46fc-ac1d-2faeb111acaf service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:00.384582 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9383f82-2c0d-46fc-ac1d-2faeb111acaf service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:00.384859 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9383f82-2c0d-46fc-ac1d-2faeb111acaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:00.386544 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c9383f82-2c0d-46fc-ac1d-2faeb111acaf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:00.475054 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9383f82-2c0d-46fc-ac1d-2faeb111acaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:58:00.476692 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9383f82-2c0d-46fc-ac1d-2faeb111acaf service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:58:00.494191 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 728/1440] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:58:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:58:02.442290 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1943aa-acd2-403c-bb24-5dd5eeae9e66 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:02.442858 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1943aa-acd2-403c-bb24-5dd5eeae9e66 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:02.442858 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1943aa-acd2-403c-bb24-5dd5eeae9e66 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:02.444253 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 713/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 15:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:58:02.449418 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a25e42-7edf-4860-b4f6-eb1e126b58dc None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:02.449883 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a25e42-7edf-4860-b4f6-eb1e126b58dc None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:02.450003 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a25e42-7edf-4860-b4f6-eb1e126b58dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:02.490200 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:58:02.515700 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-17a25e42-7edf-4860-b4f6-eb1e126b58dc None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:58:02.516890 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-17a25e42-7edf-4860-b4f6-eb1e126b58dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:02.613967 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 729/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 15:58:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:58:02.654783 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1943aa-acd2-403c-bb24-5dd5eeae9e66 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:58:02.656351 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5c1943aa-acd2-403c-bb24-5dd5eeae9e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:02.759811 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1943aa-acd2-403c-bb24-5dd5eeae9e66 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:58:02.790797 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:58:02.831607 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc0c13-699f-4395-acc0-18c9be54c94f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:58:02.832253 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc0c13-699f-4395-acc0-18c9be54c94f service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:02.832538 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc0c13-699f-4395-acc0-18c9be54c94f service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:02.832815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc0c13-699f-4395-acc0-18c9be54c94f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:02.834487 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4acc0c13-699f-4395-acc0-18c9be54c94f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:02.944489 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acc0c13-699f-4395-acc0-18c9be54c94f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:58:02.946009 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acc0c13-699f-4395-acc0-18c9be54c94f service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:58:02.964502 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 714/1443] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:58:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:58:05.221738 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d0420-3c6c-4560-8190-e49e631a0434 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:05.222299 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d0420-3c6c-4560-8190-e49e631a0434 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:05.222299 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d0420-3c6c-4560-8190-e49e631a0434 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:05.223707 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 730/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 15:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:58:05.228799 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c72e9f-b344-4d49-b2b9-53b21fa4d1df None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:05.229221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c72e9f-b344-4d49-b2b9-53b21fa4d1df None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:05.229361 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c72e9f-b344-4d49-b2b9-53b21fa4d1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:05.294386 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-45c72e9f-b344-4d49-b2b9-53b21fa4d1df None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:58:05.295594 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-45c72e9f-b344-4d49-b2b9-53b21fa4d1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:05.308581 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:58:05.397804 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 715/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 15:58:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:58:05.435337 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d0420-3c6c-4560-8190-e49e631a0434 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:58:05.436935 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-561d0420-3c6c-4560-8190-e49e631a0434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:05.537864 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d0420-3c6c-4560-8190-e49e631a0434 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:58:05.550164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:58:05.611864 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322bbe87-947c-4f56-ab7f-89d2d56da30b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:58:05.612512 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bbe87-947c-4f56-ab7f-89d2d56da30b service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:05.612799 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bbe87-947c-4f56-ab7f-89d2d56da30b service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:05.613079 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bbe87-947c-4f56-ab7f-89d2d56da30b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:05.614770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-322bbe87-947c-4f56-ab7f-89d2d56da30b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:05.720158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322bbe87-947c-4f56-ab7f-89d2d56da30b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:58:05.721910 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322bbe87-947c-4f56-ab7f-89d2d56da30b service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:58:05.740146 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 731/1446] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:58:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:58:08.072538 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa8807-1ef5-4b1a-9bfc-cf4c4937c199 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:08.073066 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa8807-1ef5-4b1a-9bfc-cf4c4937c199 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:08.073412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa8807-1ef5-4b1a-9bfc-cf4c4937c199 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:08.075101 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 716/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 15:58:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:58:08.082096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e221b2e-f221-42a2-b484-08171449dd9a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:08.082554 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e221b2e-f221-42a2-b484-08171449dd9a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:08.082663 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e221b2e-f221-42a2-b484-08171449dd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:08.129393 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:58:08.146523 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5e221b2e-f221-42a2-b484-08171449dd9a None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:58:08.147747 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5e221b2e-f221-42a2-b484-08171449dd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:08.244587 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 732/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 15:58:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:58:08.283014 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa8807-1ef5-4b1a-9bfc-cf4c4937c199 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:58:08.284605 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b8fa8807-1ef5-4b1a-9bfc-cf4c4937c199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:08.394876 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa8807-1ef5-4b1a-9bfc-cf4c4937c199 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:58:08.441168 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:58:08.475607 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5919b751-b031-456a-9922-1a33c8c02e0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:58:08.476257 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919b751-b031-456a-9922-1a33c8c02e0a service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:08.476545 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919b751-b031-456a-9922-1a33c8c02e0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:08.476825 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919b751-b031-456a-9922-1a33c8c02e0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:08.478514 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5919b751-b031-456a-9922-1a33c8c02e0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:08.584353 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5919b751-b031-456a-9922-1a33c8c02e0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:58:08.585764 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5919b751-b031-456a-9922-1a33c8c02e0a service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:58:08.603902 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 717/1449] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:58:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:58:10.756083 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7244a3f-3408-4d9a-8a29-58665e8332ac None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:10.756611 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7244a3f-3408-4d9a-8a29-58665e8332ac None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:10.756753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7244a3f-3408-4d9a-8a29-58665e8332ac None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:10.758082 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 733/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 15:58:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:58:10.763430 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1c5c5-1d74-452a-ae59-793d4d433426 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:10.763828 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1c5c5-1d74-452a-ae59-793d4d433426 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:10.764026 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1c5c5-1d74-452a-ae59-793d4d433426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:10.827443 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-fab1c5c5-1d74-452a-ae59-793d4d433426 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:58:10.829147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fab1c5c5-1d74-452a-ae59-793d4d433426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:10.851609 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:58:10.929703 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 718/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 15:58:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:58:10.968803 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7244a3f-3408-4d9a-8a29-58665e8332ac None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:58:10.970393 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b7244a3f-3408-4d9a-8a29-58665e8332ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:11.074018 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7244a3f-3408-4d9a-8a29-58665e8332ac None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:58:11.094142 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:58:11.147306 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ef8dd-53d6-4dda-b7ca-6c2f5aa42e14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:58:11.147953 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ef8dd-53d6-4dda-b7ca-6c2f5aa42e14 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:11.148237 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ef8dd-53d6-4dda-b7ca-6c2f5aa42e14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:11.148511 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ef8dd-53d6-4dda-b7ca-6c2f5aa42e14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:11.150185 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c06ef8dd-53d6-4dda-b7ca-6c2f5aa42e14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:11.259062 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ef8dd-53d6-4dda-b7ca-6c2f5aa42e14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:58:11.260567 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ef8dd-53d6-4dda-b7ca-6c2f5aa42e14 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:58:11.278427 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 734/1452] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:58:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:58:13.352656 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea77b78-c356-4816-ab24-a123bfd93904 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:13.353214 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea77b78-c356-4816-ab24-a123bfd93904 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:13.353214 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea77b78-c356-4816-ab24-a123bfd93904 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:13.354700 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 719/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 15:58:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:58:13.361822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca86f9-3c9c-41cb-82b2-3021b8a08532 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:13.362275 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca86f9-3c9c-41cb-82b2-3021b8a08532 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:13.362392 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca86f9-3c9c-41cb-82b2-3021b8a08532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:13.424394 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:58:13.428760 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b2ca86f9-3c9c-41cb-82b2-3021b8a08532 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:58:13.430680 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b2ca86f9-3c9c-41cb-82b2-3021b8a08532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:13.525155 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 735/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 15:58:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:58:13.563514 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea77b78-c356-4816-ab24-a123bfd93904 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:58:13.565092 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0ea77b78-c356-4816-ab24-a123bfd93904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:13.672970 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea77b78-c356-4816-ab24-a123bfd93904 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:58:13.720868 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:58:13.745351 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bcaf86-1ca2-4f1f-a1f7-5497ed26ce90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:58:13.746038 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bcaf86-1ca2-4f1f-a1f7-5497ed26ce90 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:13.746330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bcaf86-1ca2-4f1f-a1f7-5497ed26ce90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:13.746612 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bcaf86-1ca2-4f1f-a1f7-5497ed26ce90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:13.748259 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-42bcaf86-1ca2-4f1f-a1f7-5497ed26ce90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:13.854717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bcaf86-1ca2-4f1f-a1f7-5497ed26ce90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:58:13.856085 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bcaf86-1ca2-4f1f-a1f7-5497ed26ce90 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:58:13.873759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 720/1455] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:58:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:58:16.004420 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e51f65-68ff-4c7f-969f-f25d6a941f65 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:16.004910 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e51f65-68ff-4c7f-969f-f25d6a941f65 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:16.005020 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e51f65-68ff-4c7f-969f-f25d6a941f65 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:16.006422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 736/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 15:58:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:58:16.011626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce09f8b-2472-4cc8-9b24-acec0a8892e2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:16.012001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce09f8b-2472-4cc8-9b24-acec0a8892e2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:16.012199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce09f8b-2472-4cc8-9b24-acec0a8892e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:16.075643 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2ce09f8b-2472-4cc8-9b24-acec0a8892e2 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:58:16.076853 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2ce09f8b-2472-4cc8-9b24-acec0a8892e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:16.106187 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:58:16.173947 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 721/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 15:58:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:58:16.211713 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e51f65-68ff-4c7f-969f-f25d6a941f65 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:58:16.213297 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-15e51f65-68ff-4c7f-969f-f25d6a941f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:16.319444 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e51f65-68ff-4c7f-969f-f25d6a941f65 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:58:16.348085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:58:16.393213 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd18ca-970e-4aea-882c-67f1ce63534a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:58:16.393886 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd18ca-970e-4aea-882c-67f1ce63534a service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:16.394175 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd18ca-970e-4aea-882c-67f1ce63534a service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:16.394455 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd18ca-970e-4aea-882c-67f1ce63534a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:16.396109 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f7cd18ca-970e-4aea-882c-67f1ce63534a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:16.504350 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cd18ca-970e-4aea-882c-67f1ce63534a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:58:16.505746 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cd18ca-970e-4aea-882c-67f1ce63534a service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:58:16.523989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 737/1458] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:58:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:58:18.600241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6add722-fdb2-47ea-b65a-c55b5d415cee None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:18.600843 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6add722-fdb2-47ea-b65a-c55b5d415cee None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:18.600843 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6add722-fdb2-47ea-b65a-c55b5d415cee None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:18.602118 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 722/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 15:58:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:58:18.607403 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6bc6da-f6fc-414c-b389-02443a5e341a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:18.607830 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6bc6da-f6fc-414c-b389-02443a5e341a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:18.607947 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6bc6da-f6fc-414c-b389-02443a5e341a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:18.685793 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4d6bc6da-f6fc-414c-b389-02443a5e341a None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:58:18.687056 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4d6bc6da-f6fc-414c-b389-02443a5e341a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:18.695849 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:58:18.784723 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 738/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 15:58:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:58:18.822175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6add722-fdb2-47ea-b65a-c55b5d415cee None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:58:18.823707 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a6add722-fdb2-47ea-b65a-c55b5d415cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:18.927125 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6add722-fdb2-47ea-b65a-c55b5d415cee None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:58:18.982045 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:58:19.000200 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538505ca-8502-4398-9d55-5762a3547bf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:58:19.000873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538505ca-8502-4398-9d55-5762a3547bf7 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:19.001152 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538505ca-8502-4398-9d55-5762a3547bf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:19.001424 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538505ca-8502-4398-9d55-5762a3547bf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:19.003112 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-538505ca-8502-4398-9d55-5762a3547bf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:19.110725 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538505ca-8502-4398-9d55-5762a3547bf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:58:19.112286 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538505ca-8502-4398-9d55-5762a3547bf7 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:58:19.130737 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 723/1461] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:58:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:58:21.204901 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0ff87-8c60-497f-9db0-cb816b0976a1 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:21.205492 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0ff87-8c60-497f-9db0-cb816b0976a1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:21.205492 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0ff87-8c60-497f-9db0-cb816b0976a1 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:21.206862 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 739/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 15:58:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:58:21.212194 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2713bde-6841-4f3e-b33b-942658feae11 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:21.212625 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2713bde-6841-4f3e-b33b-942658feae11 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:21.212773 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2713bde-6841-4f3e-b33b-942658feae11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:21.278266 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f2713bde-6841-4f3e-b33b-942658feae11 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:58:21.279474 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f2713bde-6841-4f3e-b33b-942658feae11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:21.317253 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:58:21.374965 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 724/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 15:58:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:58:21.414030 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0ff87-8c60-497f-9db0-cb816b0976a1 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:58:21.415560 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-85f0ff87-8c60-497f-9db0-cb816b0976a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:21.521617 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0ff87-8c60-497f-9db0-cb816b0976a1 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:58:21.558560 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:58:21.593885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d195885-223d-4e48-83a9-aaec5b8a2659 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:58:21.594527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d195885-223d-4e48-83a9-aaec5b8a2659 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:21.594808 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d195885-223d-4e48-83a9-aaec5b8a2659 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:21.595084 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d195885-223d-4e48-83a9-aaec5b8a2659 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:21.596981 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7d195885-223d-4e48-83a9-aaec5b8a2659 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:21.701029 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d195885-223d-4e48-83a9-aaec5b8a2659 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:58:21.703079 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d195885-223d-4e48-83a9-aaec5b8a2659 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:58:21.720411 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 740/1464] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:58:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:58:23.807631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b3aa1-789a-4ecb-9eac-ada916642e92 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:23.808207 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b3aa1-789a-4ecb-9eac-ada916642e92 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:23.808207 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b3aa1-789a-4ecb-9eac-ada916642e92 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:23.809611 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 725/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 15:58:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:58:23.815970 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91d18a-b330-491e-95c4-f2ee5be7695e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:23.816498 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91d18a-b330-491e-95c4-f2ee5be7695e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:23.816658 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91d18a-b330-491e-95c4-f2ee5be7695e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:23.885053 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9e91d18a-b330-491e-95c4-f2ee5be7695e None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:58:23.886295 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9e91d18a-b330-491e-95c4-f2ee5be7695e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:23.904677 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:58:23.983769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 741/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 15:58:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:58:24.021177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3b3aa1-789a-4ecb-9eac-ada916642e92 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:58:24.022742 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2a3b3aa1-789a-4ecb-9eac-ada916642e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:24.133663 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3b3aa1-789a-4ecb-9eac-ada916642e92 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:58:24.204250 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:58:24.211310 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dfd6ba-3b25-4cea-b17f-6918c6ac8b8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:58:24.211955 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dfd6ba-3b25-4cea-b17f-6918c6ac8b8f service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:24.212236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dfd6ba-3b25-4cea-b17f-6918c6ac8b8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:24.212511 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dfd6ba-3b25-4cea-b17f-6918c6ac8b8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:24.214167 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e5dfd6ba-3b25-4cea-b17f-6918c6ac8b8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:24.323955 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dfd6ba-3b25-4cea-b17f-6918c6ac8b8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:58:24.325397 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dfd6ba-3b25-4cea-b17f-6918c6ac8b8f service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:58:24.343220 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 726/1467] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:58:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:58:26.382727 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2683d849-2ec7-41ba-a876-0b3c2df19970 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:26.383322 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2683d849-2ec7-41ba-a876-0b3c2df19970 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:26.383322 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2683d849-2ec7-41ba-a876-0b3c2df19970 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:26.384653 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 742/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 15:58:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 15:58:26.390571 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b9aa5-6e15-4e33-9d39-8ced5738fdff None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:26.390986 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b9aa5-6e15-4e33-9d39-8ced5738fdff None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:26.391113 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b9aa5-6e15-4e33-9d39-8ced5738fdff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:26.453983 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3d7b9aa5-6e15-4e33-9d39-8ced5738fdff None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:58:26.455254 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3d7b9aa5-6e15-4e33-9d39-8ced5738fdff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:26.502383 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:58:26.550766 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 727/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 15:58:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:58:26.588441 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2683d849-2ec7-41ba-a876-0b3c2df19970 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:58:26.590128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2683d849-2ec7-41ba-a876-0b3c2df19970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:26.693746 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2683d849-2ec7-41ba-a876-0b3c2df19970 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:58:26.740190 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:58:26.766180 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a72815-22c0-4af7-bc11-439a37157736 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:58:26.766756 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a72815-22c0-4af7-bc11-439a37157736 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:26.767037 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a72815-22c0-4af7-bc11-439a37157736 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:26.767312 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a72815-22c0-4af7-bc11-439a37157736 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:26.768959 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-01a72815-22c0-4af7-bc11-439a37157736 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:26.876328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a72815-22c0-4af7-bc11-439a37157736 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:58:26.877783 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a72815-22c0-4af7-bc11-439a37157736 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:58:26.895441 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 743/1470] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:58:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:58:36.548914 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=22 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:58:36.893259 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=66 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:58:46.658193 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37561bcd-ae6a-4b18-894e-b7be3f834174 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:46.658757 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37561bcd-ae6a-4b18-894e-b7be3f834174 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:46.658757 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37561bcd-ae6a-4b18-894e-b7be3f834174 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:46.660140 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 728/1471] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:58:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:58:46.665272 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f61e0-6259-47ce-bd78-5d284e5742ff None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:46.665701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f61e0-6259-47ce-bd78-5d284e5742ff None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:46.665861 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f61e0-6259-47ce-bd78-5d284e5742ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:46.730989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d26f61e0-6259-47ce-bd78-5d284e5742ff None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:58:46.732215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d26f61e0-6259-47ce-bd78-5d284e5742ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:46.831646 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 744/1472] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:58:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:58:46.917557 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c780c-0482-426c-81ae-8ae68b32c1b2 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:46.917880 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c780c-0482-426c-81ae-8ae68b32c1b2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:46.918172 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c780c-0482-426c-81ae-8ae68b32c1b2 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:46.919533 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 729/1473] 10.222.0.52 () {54 vars in 966 bytes} [Thu Jul 21 15:58:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 15:58:46.924951 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abf8bf-71e0-445a-837c-367a900df8df None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:46.925268 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abf8bf-71e0-445a-837c-367a900df8df None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:46.925541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abf8bf-71e0-445a-837c-367a900df8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:47.005906 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-15abf8bf-71e0-445a-837c-367a900df8df None None] MFA Rules not processed for user `d73a7d5f37e144768d7cc53f13cd5157`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:58:47.007101 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-15abf8bf-71e0-445a-837c-367a900df8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:47.026885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:58:47.083011 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 745/1474] 10.222.0.52 () {58 vars in 1063 bytes} [Thu Jul 21 15:58:46 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:58:47.090515 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c780c-0482-426c-81ae-8ae68b32c1b2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:58:47.092044 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-816c780c-0482-426c-81ae-8ae68b32c1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:47.193946 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c780c-0482-426c-81ae-8ae68b32c1b2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:58:47.266986 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a64e55-2686-46e5-871a-dc0b3b8c570e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73a7d5f37e144768d7cc53f13cd5157', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:58:47.267632 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a64e55-2686-46e5-871a-dc0b3b8c570e service glance] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:47.267916 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a64e55-2686-46e5-871a-dc0b3b8c570e service glance] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:47.268193 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a64e55-2686-46e5-871a-dc0b3b8c570e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:47.269869 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d8a64e55-2686-46e5-871a-dc0b3b8c570e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:47.322784 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:58:47.378228 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a64e55-2686-46e5-871a-dc0b3b8c570e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:58:47.379676 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a64e55-2686-46e5-871a-dc0b3b8c570e service glance] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:58:47.398988 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 730/1475] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:58:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:58:49.528531 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57f612-88f7-4ad3-8997-74a9c583272a None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:49.529211 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57f612-88f7-4ad3-8997-74a9c583272a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:49.529211 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57f612-88f7-4ad3-8997-74a9c583272a None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:49.530461 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 746/1476] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:58:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:58:49.535677 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970323f4-07cc-4b0c-8ea6-b4c8c4c3b01b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:49.536058 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970323f4-07cc-4b0c-8ea6-b4c8c4c3b01b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:49.536268 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970323f4-07cc-4b0c-8ea6-b4c8c4c3b01b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:49.604925 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-970323f4-07cc-4b0c-8ea6-b4c8c4c3b01b None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:58:49.606209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-970323f4-07cc-4b0c-8ea6-b4c8c4c3b01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:49.702919 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 731/1477] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:58:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:58:49.726735 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57f612-88f7-4ad3-8997-74a9c583272a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:58:49.728269 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6c57f612-88f7-4ad3-8997-74a9c583272a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:49.852422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57f612-88f7-4ad3-8997-74a9c583272a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:58:49.901477 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:58:49.946008 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ce100-b925-455a-8831-fbf3beaf91d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:58:49.946666 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ce100-b925-455a-8831-fbf3beaf91d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:49.947060 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ce100-b925-455a-8831-fbf3beaf91d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:49.947353 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ce100-b925-455a-8831-fbf3beaf91d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:49.948069 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ce100-b925-455a-8831-fbf3beaf91d6 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:58:49.949898 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-2e6ce100-b925-455a-8831-fbf3beaf91d6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 15:58:49.949898 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: warnings.warn(msg) Jul 21 15:58:49.949898 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: Jul 21 15:58:49.951075 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ce100-b925-455a-8831-fbf3beaf91d6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:58:49.969291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 747/1478] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 15:58:49 2022] GET /identity/v3/projects => generated 2553 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 15:58:52.033162 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec404d14-b71a-4274-8be1-f7f45aa98642 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:52.033721 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec404d14-b71a-4274-8be1-f7f45aa98642 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:52.033841 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec404d14-b71a-4274-8be1-f7f45aa98642 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:52.035119 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 732/1479] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:58:52.040495 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205340cc-888c-43d1-a462-c27c53695c7a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:52.040885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205340cc-888c-43d1-a462-c27c53695c7a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:52.041042 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205340cc-888c-43d1-a462-c27c53695c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:52.105743 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-205340cc-888c-43d1-a462-c27c53695c7a None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:58:52.106983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-205340cc-888c-43d1-a462-c27c53695c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:52.202455 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 748/1480] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:58:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:58:52.246449 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec404d14-b71a-4274-8be1-f7f45aa98642 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:58:52.247986 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ec404d14-b71a-4274-8be1-f7f45aa98642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:52.258224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:58:52.394206 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec404d14-b71a-4274-8be1-f7f45aa98642 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:58:52.472820 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:58:52.505774 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df84e56-7d39-402e-8238-9eca265bee0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:58:52.506351 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df84e56-7d39-402e-8238-9eca265bee0a service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:52.506637 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df84e56-7d39-402e-8238-9eca265bee0a service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:52.506916 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df84e56-7d39-402e-8238-9eca265bee0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:52.508559 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2df84e56-7d39-402e-8238-9eca265bee0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:52.597166 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df84e56-7d39-402e-8238-9eca265bee0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:58:52.598817 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df84e56-7d39-402e-8238-9eca265bee0a service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:58:52.617120 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 733/1481] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:58:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:58:54.829073 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b45c7-ecf6-4299-91c0-5636df05ff79 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:54.829669 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b45c7-ecf6-4299-91c0-5636df05ff79 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:54.829669 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b45c7-ecf6-4299-91c0-5636df05ff79 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:54.831010 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 749/1482] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:58:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:58:54.836264 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955e8ef-58bd-46e7-b25f-f3f6a5850a17 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:54.836670 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955e8ef-58bd-46e7-b25f-f3f6a5850a17 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:54.836822 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955e8ef-58bd-46e7-b25f-f3f6a5850a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:54.902555 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4955e8ef-58bd-46e7-b25f-f3f6a5850a17 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:58:54.903770 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4955e8ef-58bd-46e7-b25f-f3f6a5850a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:54.948445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:58:54.999108 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 734/1483] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:58:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:58:55.044115 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b45c7-ecf6-4299-91c0-5636df05ff79 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:58:55.046192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6e1b45c7-ecf6-4299-91c0-5636df05ff79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:55.076176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:58:55.155032 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b45c7-ecf6-4299-91c0-5636df05ff79 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:58:55.226178 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c751f8-b11f-4ba1-aec4-7d45bdc15509 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:58:55.226825 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c751f8-b11f-4ba1-aec4-7d45bdc15509 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:55.227112 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c751f8-b11f-4ba1-aec4-7d45bdc15509 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:55.227389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c751f8-b11f-4ba1-aec4-7d45bdc15509 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:55.229047 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d7c751f8-b11f-4ba1-aec4-7d45bdc15509 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:55.305799 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:58:55.340642 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c751f8-b11f-4ba1-aec4-7d45bdc15509 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:58:55.342909 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c751f8-b11f-4ba1-aec4-7d45bdc15509 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:58:55.364094 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 750/1484] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:58:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:58:57.432538 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e679d1-5bec-4df4-b1e0-66cd81d38581 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:57.433000 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e679d1-5bec-4df4-b1e0-66cd81d38581 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:57.433106 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e679d1-5bec-4df4-b1e0-66cd81d38581 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:57.434502 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 735/1485] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:58:57.440551 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afa23b-cc83-4242-adc4-a5738e1a11cc None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:57.441000 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afa23b-cc83-4242-adc4-a5738e1a11cc None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:57.441158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afa23b-cc83-4242-adc4-a5738e1a11cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:57.504772 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-01afa23b-cc83-4242-adc4-a5738e1a11cc None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:58:57.505927 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-01afa23b-cc83-4242-adc4-a5738e1a11cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:57.601020 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 751/1486] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:58:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:58:57.639275 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e679d1-5bec-4df4-b1e0-66cd81d38581 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:58:57.640810 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-71e679d1-5bec-4df4-b1e0-66cd81d38581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:57.744457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e679d1-5bec-4df4-b1e0-66cd81d38581 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:58:57.816991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:58:57.818747 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb5b70-36ef-4ab3-99fb-6097fee1fbb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:58:57.819376 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb5b70-36ef-4ab3-99fb-6097fee1fbb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:58:57.819857 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb5b70-36ef-4ab3-99fb-6097fee1fbb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:58:57.820339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb5b70-36ef-4ab3-99fb-6097fee1fbb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:58:57.822216 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7fdb5b70-36ef-4ab3-99fb-6097fee1fbb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:58:57.930261 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdb5b70-36ef-4ab3-99fb-6097fee1fbb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:58:57.931704 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdb5b70-36ef-4ab3-99fb-6097fee1fbb8 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:58:57.949951 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 736/1487] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:58:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:59:00.121772 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071aacf-5f92-4e36-9082-a183991354a5 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:59:00.122319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071aacf-5f92-4e36-9082-a183991354a5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:59:00.122319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071aacf-5f92-4e36-9082-a183991354a5 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:59:00.123730 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 752/1488] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:59:00.129089 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7e44c-0f1a-4b0b-bd20-835bcfbcfb8f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:59:00.129538 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7e44c-0f1a-4b0b-bd20-835bcfbcfb8f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:59:00.129736 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7e44c-0f1a-4b0b-bd20-835bcfbcfb8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:59:00.192939 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f0f7e44c-0f1a-4b0b-bd20-835bcfbcfb8f None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:59:00.194194 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f0f7e44c-0f1a-4b0b-bd20-835bcfbcfb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:59:00.247620 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:59:00.408837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 737/1489] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:59:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:59:00.449804 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071aacf-5f92-4e36-9082-a183991354a5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:59:00.451371 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8071aacf-5f92-4e36-9082-a183991354a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:59:00.486745 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:59:00.555962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071aacf-5f92-4e36-9082-a183991354a5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:59:00.629820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adca465-88d2-4b1b-8d21-1c60459b8753 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:59:00.630452 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adca465-88d2-4b1b-8d21-1c60459b8753 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:59:00.630734 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adca465-88d2-4b1b-8d21-1c60459b8753 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:59:00.631007 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adca465-88d2-4b1b-8d21-1c60459b8753 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:59:00.632738 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4adca465-88d2-4b1b-8d21-1c60459b8753 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:59:00.710829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:59:00.742255 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adca465-88d2-4b1b-8d21-1c60459b8753 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:59:00.743706 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adca465-88d2-4b1b-8d21-1c60459b8753 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:59:00.762756 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 753/1490] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:59:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:59:02.957799 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c2a34-5b1a-4788-9db7-deda9e22320f None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:59:02.958335 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c2a34-5b1a-4788-9db7-deda9e22320f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:59:02.958335 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c2a34-5b1a-4788-9db7-deda9e22320f None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:59:02.959695 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 738/1491] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 15:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 15:59:02.964825 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96580e1-2689-475d-b03e-2e50515dcc11 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:59:02.965264 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96580e1-2689-475d-b03e-2e50515dcc11 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:59:02.965395 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96580e1-2689-475d-b03e-2e50515dcc11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:59:03.029030 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e96580e1-2689-475d-b03e-2e50515dcc11 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 15:59:03.030770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e96580e1-2689-475d-b03e-2e50515dcc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:59:03.127745 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 754/1492] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 15:59:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 15:59:03.166114 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c2a34-5b1a-4788-9db7-deda9e22320f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 15:59:03.167646 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fd7c2a34-5b1a-4788-9db7-deda9e22320f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:59:03.275520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c2a34-5b1a-4788-9db7-deda9e22320f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 15:59:03.354463 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e178314-741b-4bc8-979b-670b8affd98f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 15:59:03.355107 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e178314-741b-4bc8-979b-670b8affd98f service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 15:59:03.355390 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e178314-741b-4bc8-979b-670b8affd98f service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 15:59:03.355666 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e178314-741b-4bc8-979b-670b8affd98f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 15:59:03.357309 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4e178314-741b-4bc8-979b-670b8affd98f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 15:59:03.366370 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:59:03.469564 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e178314-741b-4bc8-979b-670b8affd98f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 15:59:03.471070 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e178314-741b-4bc8-979b-670b8affd98f service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 15:59:03.489174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 739/1493] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 15:59:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 15:59:13.125914 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=82 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 15:59:13.486861 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=50 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:00:35.818965 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1965d-a33b-4013-a5f9-9940fcb8286a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:00:35.819559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1965d-a33b-4013-a5f9-9940fcb8286a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:00:35.819559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1965d-a33b-4013-a5f9-9940fcb8286a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:00:35.898017 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7bb1965d-a33b-4013-a5f9-9940fcb8286a None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:00:35.899229 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7bb1965d-a33b-4013-a5f9-9940fcb8286a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:00:35.993839 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 755/1494] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:00:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:00:36.002479 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd60315-c296-46f9-b72b-c5d69af3fb2b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:00:36.002963 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd60315-c296-46f9-b72b-c5d69af3fb2b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:00:36.003073 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd60315-c296-46f9-b72b-c5d69af3fb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:00:36.075143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0dd60315-c296-46f9-b72b-c5d69af3fb2b None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:00:36.076346 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0dd60315-c296-46f9-b72b-c5d69af3fb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:00:36.170952 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 740/1495] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:00:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:00:36.175756 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1965d-a33b-4013-a5f9-9940fcb8286a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:00:36.176991 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7bb1965d-a33b-4013-a5f9-9940fcb8286a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:00:36.189221 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1965d-a33b-4013-a5f9-9940fcb8286a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:00:36.252140 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:00:36.285381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffaf00a-0c9d-4a29-920d-550644bdc2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:00:36.286078 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffaf00a-0c9d-4a29-920d-550644bdc2de admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:00:36.286364 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffaf00a-0c9d-4a29-920d-550644bdc2de admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:00:36.286644 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffaf00a-0c9d-4a29-920d-550644bdc2de admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:00:36.287365 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffaf00a-0c9d-4a29-920d-550644bdc2de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:00:36.287675 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffaf00a-0c9d-4a29-920d-550644bdc2de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:00:36.288706 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-bffaf00a-0c9d-4a29-920d-550644bdc2de admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 16:00:36.288706 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: warnings.warn(msg) Jul 21 16:00:36.288706 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: Jul 21 16:00:36.289618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffaf00a-0c9d-4a29-920d-550644bdc2de admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:00:36.301496 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 756/1496] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:00:36.310377 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd60315-c296-46f9-b72b-c5d69af3fb2b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:00:36.319462 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd60315-c296-46f9-b72b-c5d69af3fb2b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:00:36.379434 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:00:36.411800 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b087e1-2dda-440b-8a43-d63e6d9a6627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:00:36.412450 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b087e1-2dda-440b-8a43-d63e6d9a6627 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:00:36.412731 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b087e1-2dda-440b-8a43-d63e6d9a6627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:00:36.413005 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b087e1-2dda-440b-8a43-d63e6d9a6627 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:00:36.413739 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b087e1-2dda-440b-8a43-d63e6d9a6627 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:00:36.415492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: WARNING py.warnings [None req-f8b087e1-2dda-440b-8a43-d63e6d9a6627 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 16:00:36.415492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: warnings.warn(msg) Jul 21 16:00:36.415492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: Jul 21 16:00:36.416673 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b087e1-2dda-440b-8a43-d63e6d9a6627 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:00:36.470062 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 741/1497] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:00:36 2022] POST /identity/v3/projects => generated 377 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:00:36.474685 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffaf00a-0c9d-4a29-920d-550644bdc2de admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:00:36.476213 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bffaf00a-0c9d-4a29-920d-550644bdc2de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:00:36.510664 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffaf00a-0c9d-4a29-920d-550644bdc2de admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:00:36.618521 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f302b18-fa9d-4b78-abc9-4b213ceb8bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:00:36.619191 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f302b18-fa9d-4b78-abc9-4b213ceb8bdc admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:00:36.619473 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f302b18-fa9d-4b78-abc9-4b213ceb8bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:00:36.619748 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f302b18-fa9d-4b78-abc9-4b213ceb8bdc admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:00:36.620451 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f302b18-fa9d-4b78-abc9-4b213ceb8bdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:00:36.621468 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-8f302b18-fa9d-4b78-abc9-4b213ceb8bdc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 16:00:36.621468 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: warnings.warn(msg) Jul 21 16:00:36.621468 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: Jul 21 16:00:36.622705 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f302b18-fa9d-4b78-abc9-4b213ceb8bdc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:00:36.662241 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 757/1498] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:00:36 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:00:36.666263 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b087e1-2dda-440b-8a43-d63e6d9a6627 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:00:36.668527 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f8b087e1-2dda-440b-8a43-d63e6d9a6627 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:00:36.729552 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b087e1-2dda-440b-8a43-d63e6d9a6627 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:00:36.809447 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:00:36.820982 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b154949-ecc1-4cbf-a22f-3781587e9add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:00:36.821670 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b154949-ecc1-4cbf-a22f-3781587e9add admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:00:36.821952 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b154949-ecc1-4cbf-a22f-3781587e9add admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:00:36.822223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b154949-ecc1-4cbf-a22f-3781587e9add admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:00:36.822944 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b154949-ecc1-4cbf-a22f-3781587e9add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:00:36.823983 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: WARNING py.warnings [None req-5b154949-ecc1-4cbf-a22f-3781587e9add admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 16:00:36.823983 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: warnings.warn(msg) Jul 21 16:00:36.823983 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: Jul 21 16:00:36.824857 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b154949-ecc1-4cbf-a22f-3781587e9add admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:00:36.836837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 742/1499] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:00:36.842021 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f302b18-fa9d-4b78-abc9-4b213ceb8bdc admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:00:36.843550 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8f302b18-fa9d-4b78-abc9-4b213ceb8bdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:00:36.856266 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f302b18-fa9d-4b78-abc9-4b213ceb8bdc admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:00:36.882558 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:00:36.956520 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3bf0b8-6002-479d-8e97-b990e313b60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:00:36.957362 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bf0b8-6002-479d-8e97-b990e313b60f admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:00:36.957692 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bf0b8-6002-479d-8e97-b990e313b60f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:00:36.958065 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bf0b8-6002-479d-8e97-b990e313b60f admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:00:36.958863 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3bf0b8-6002-479d-8e97-b990e313b60f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:00:36.960227 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-cf3bf0b8-6002-479d-8e97-b990e313b60f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 16:00:36.960227 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: warnings.warn(msg) Jul 21 16:00:36.960227 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: Jul 21 16:00:36.961249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3bf0b8-6002-479d-8e97-b990e313b60f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:00:36.974673 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 758/1500] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:00:36.979147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b154949-ecc1-4cbf-a22f-3781587e9add admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:00:36.981177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5b154949-ecc1-4cbf-a22f-3781587e9add admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:00:37.014452 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b154949-ecc1-4cbf-a22f-3781587e9add admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:00:37.107607 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9becf088-edbd-4aa8-87ba-7fcd810ccc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:00:37.108487 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9becf088-edbd-4aa8-87ba-7fcd810ccc21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:00:37.108776 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9becf088-edbd-4aa8-87ba-7fcd810ccc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:00:37.109057 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9becf088-edbd-4aa8-87ba-7fcd810ccc21 admin admin] PATH_INFO: `/v3/projects/e212a9e75a5a4d298423b50116169913/users/6fb56b59a9a443c88d9cf8f3486742b4/roles/54195e660fc349978b39d5794765a095` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:00:37.131052 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9becf088-edbd-4aa8-87ba-7fcd810ccc21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e212a9e75a5a4d298423b50116169913, user_id=6fb56b59a9a443c88d9cf8f3486742b4, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:00:37.132101 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: WARNING py.warnings [None req-9becf088-edbd-4aa8-87ba-7fcd810ccc21 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 16:00:37.132101 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: warnings.warn(msg) Jul 21 16:00:37.132101 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: Jul 21 16:00:37.134056 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9becf088-edbd-4aa8-87ba-7fcd810ccc21 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:00:37.158471 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 743/1501] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:00:36 2022] PUT /identity/v3/projects/e212a9e75a5a4d298423b50116169913/users/6fb56b59a9a443c88d9cf8f3486742b4/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:00:37.171069 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a888f-ee3d-4d36-bda9-4121a62cb5a1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:00:37.171364 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a888f-ee3d-4d36-bda9-4121a62cb5a1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:00:37.171657 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a888f-ee3d-4d36-bda9-4121a62cb5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:00:37.226655 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-be4a888f-ee3d-4d36-bda9-4121a62cb5a1 None None] MFA Rules not processed for user `6fb56b59a9a443c88d9cf8f3486742b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:00:37.227858 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-be4a888f-ee3d-4d36-bda9-4121a62cb5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:00:37.304931 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 759/1502] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:00:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:00:37.318555 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76550845-aaf6-4194-b93e-90b94a62115a None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:00:37.318844 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76550845-aaf6-4194-b93e-90b94a62115a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:00:37.319119 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76550845-aaf6-4194-b93e-90b94a62115a None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:00:37.320475 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 744/1503] 10.222.0.52 () {54 vars in 964 bytes} [Thu Jul 21 16:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 16:00:37.325775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023c39d-84a4-45e8-b1b5-e982ffa58713 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:00:37.326065 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023c39d-84a4-45e8-b1b5-e982ffa58713 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:00:37.326368 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023c39d-84a4-45e8-b1b5-e982ffa58713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:00:37.355394 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:00:37.400539 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8023c39d-84a4-45e8-b1b5-e982ffa58713 None None] MFA Rules not processed for user `87c0598af7714aeba08fffbb6805a49e`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:00:37.401822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8023c39d-84a4-45e8-b1b5-e982ffa58713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:00:37.476132 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 760/1504] 10.222.0.52 () {58 vars in 1061 bytes} [Thu Jul 21 16:00:37 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:00:37.483839 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76550845-aaf6-4194-b93e-90b94a62115a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:00:37.485337 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-76550845-aaf6-4194-b93e-90b94a62115a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:00:37.534868 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:00:37.587803 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76550845-aaf6-4194-b93e-90b94a62115a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:00:37.659944 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb20dde-be84-4a3d-9e03-27306bba4164 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:00:37.660590 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb20dde-be84-4a3d-9e03-27306bba4164 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:00:37.660876 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb20dde-be84-4a3d-9e03-27306bba4164 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:00:37.661157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb20dde-be84-4a3d-9e03-27306bba4164 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:00:37.662815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7eb20dde-be84-4a3d-9e03-27306bba4164 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:00:37.758976 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb20dde-be84-4a3d-9e03-27306bba4164 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:00:37.760337 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb20dde-be84-4a3d-9e03-27306bba4164 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:00:37.763847 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:00:37.778461 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 745/1505] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:00:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:00:37.820589 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36dac5-c595-4e93-b13e-a5130a575d33 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:00:37.820883 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36dac5-c595-4e93-b13e-a5130a575d33 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:00:37.821165 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36dac5-c595-4e93-b13e-a5130a575d33 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:00:37.822460 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 761/1506] 10.222.0.52 () {52 vars in 829 bytes} [Thu Jul 21 16:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 16:00:37.826081 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb20dde-be84-4a3d-9e03-27306bba4164 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:00:37.827287 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7eb20dde-be84-4a3d-9e03-27306bba4164 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:00:37.933494 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb20dde-be84-4a3d-9e03-27306bba4164 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:00:38.026133 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec81d7cf-1bab-469e-9956-19aa2af06c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:00:38.026927 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81d7cf-1bab-469e-9956-19aa2af06c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:00:38.027209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81d7cf-1bab-469e-9956-19aa2af06c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:00:38.027486 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81d7cf-1bab-469e-9956-19aa2af06c4b admin admin] PATH_INFO: `/v3/users/6fb56b59a9a443c88d9cf8f3486742b4` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:00:38.039398 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec81d7cf-1bab-469e-9956-19aa2af06c4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fb56b59a9a443c88d9cf8f3486742b4)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:00:38.040429 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: WARNING py.warnings [None req-ec81d7cf-1bab-469e-9956-19aa2af06c4b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 16:00:38.040429 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: warnings.warn(msg) Jul 21 16:00:38.040429 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: Jul 21 16:00:38.041777 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec81d7cf-1bab-469e-9956-19aa2af06c4b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:00:38.051529 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:00:38.104986 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ec81d7cf-1bab-469e-9956-19aa2af06c4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fb56b59a9a443c88d9cf8f3486742b4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:00:38.105551 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ec81d7cf-1bab-469e-9956-19aa2af06c4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fb56b59a9a443c88d9cf8f3486742b4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:00:38.106007 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ec81d7cf-1bab-469e-9956-19aa2af06c4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fb56b59a9a443c88d9cf8f3486742b4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:00:38.118067 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ec81d7cf-1bab-469e-9956-19aa2af06c4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fb56b59a9a443c88d9cf8f3486742b4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:00:38.127516 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ec81d7cf-1bab-469e-9956-19aa2af06c4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fb56b59a9a443c88d9cf8f3486742b4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:00:38.150365 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 746/1507] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:00:37 2022] DELETE /identity/v3/users/6fb56b59a9a443c88d9cf8f3486742b4 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:00:38.155274 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197bb5cd-3663-4076-a8bb-77c01199306a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:00:38.155753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197bb5cd-3663-4076-a8bb-77c01199306a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:00:38.155863 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197bb5cd-3663-4076-a8bb-77c01199306a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:00:38.217013 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-197bb5cd-3663-4076-a8bb-77c01199306a None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:00:38.218255 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-197bb5cd-3663-4076-a8bb-77c01199306a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:00:38.295653 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:00:38.308994 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 762/1508] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:00:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:00:38.318523 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec81d7cf-1bab-469e-9956-19aa2af06c4b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:00:38.319796 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ec81d7cf-1bab-469e-9956-19aa2af06c4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:00:38.418978 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec81d7cf-1bab-469e-9956-19aa2af06c4b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:00:38.473116 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:00:38.493495 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c683691e-6027-42c1-9cc3-5551569170b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:00:38.494169 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683691e-6027-42c1-9cc3-5551569170b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:00:38.494452 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683691e-6027-42c1-9cc3-5551569170b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:00:38.494729 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683691e-6027-42c1-9cc3-5551569170b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:00:38.496403 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c683691e-6027-42c1-9cc3-5551569170b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:00:38.606242 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c683691e-6027-42c1-9cc3-5551569170b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:00:38.607717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c683691e-6027-42c1-9cc3-5551569170b5 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:00:38.625720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 747/1509] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:00:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:00:38.972955 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197bb5cd-3663-4076-a8bb-77c01199306a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:00:38.974260 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-197bb5cd-3663-4076-a8bb-77c01199306a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:00:38.986487 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197bb5cd-3663-4076-a8bb-77c01199306a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:00:39.089470 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda432b-0521-456d-bd2a-7cd1452ba940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:00:39.090468 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda432b-0521-456d-bd2a-7cd1452ba940 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:00:39.090760 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda432b-0521-456d-bd2a-7cd1452ba940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:00:39.091089 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda432b-0521-456d-bd2a-7cd1452ba940 admin admin] PATH_INFO: `/v3/projects/e212a9e75a5a4d298423b50116169913` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:00:39.101103 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda432b-0521-456d-bd2a-7cd1452ba940 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e212a9e75a5a4d298423b50116169913)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:00:39.102167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-cfda432b-0521-456d-bd2a-7cd1452ba940 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 16:00:39.102167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: warnings.warn(msg) Jul 21 16:00:39.102167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: Jul 21 16:00:39.103376 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda432b-0521-456d-bd2a-7cd1452ba940 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:00:39.160009 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cfda432b-0521-456d-bd2a-7cd1452ba940 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e212a9e75a5a4d298423b50116169913', 'request_id': 'req-cfda432b-0521-456d-bd2a-7cd1452ba940'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:00:39.170608 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cfda432b-0521-456d-bd2a-7cd1452ba940 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e212a9e75a5a4d298423b50116169913', 'request_id': 'req-cfda432b-0521-456d-bd2a-7cd1452ba940'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:00:39.176687 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cfda432b-0521-456d-bd2a-7cd1452ba940 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e212a9e75a5a4d298423b50116169913', 'request_id': 'req-cfda432b-0521-456d-bd2a-7cd1452ba940'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:00:39.206278 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cfda432b-0521-456d-bd2a-7cd1452ba940 admin admin] The token cache is being invalidate because project e212a9e75a5a4d298423b50116169913 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:00:39.206564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cfda432b-0521-456d-bd2a-7cd1452ba940 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:00:39.207072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cfda432b-0521-456d-bd2a-7cd1452ba940 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:00:39.208416 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 763/1510] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:00:38 2022] DELETE /identity/v3/projects/e212a9e75a5a4d298423b50116169913 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:00:48.623483 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=62 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:00:49.204310 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:02:23.803840 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767aefd-c893-441b-80e2-5bdf2708e32e None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:02:23.804442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767aefd-c893-441b-80e2-5bdf2708e32e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:02:23.804442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767aefd-c893-441b-80e2-5bdf2708e32e None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:02:23.805834 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 748/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 16:02:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 16:02:23.811219 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d9790-ea30-4709-ae66-72af2f8dbd15 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:02:23.811659 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d9790-ea30-4709-ae66-72af2f8dbd15 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:02:23.811769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d9790-ea30-4709-ae66-72af2f8dbd15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:02:24.036246 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-495d9790-ea30-4709-ae66-72af2f8dbd15 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:02:24.037485 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-495d9790-ea30-4709-ae66-72af2f8dbd15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:02:24.140827 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 764/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 16:02:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:02:24.295537 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767aefd-c893-441b-80e2-5bdf2708e32e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:02:24.297682 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c767aefd-c893-441b-80e2-5bdf2708e32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:02:24.427251 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767aefd-c893-441b-80e2-5bdf2708e32e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:02:24.433435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459ea54-8042-44cd-9255-2b3d463e9bcf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:02:24.434122 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459ea54-8042-44cd-9255-2b3d463e9bcf service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:02:24.434417 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459ea54-8042-44cd-9255-2b3d463e9bcf service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:02:24.434696 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459ea54-8042-44cd-9255-2b3d463e9bcf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:02:24.442759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459ea54-8042-44cd-9255-2b3d463e9bcf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:02:24.444347 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459ea54-8042-44cd-9255-2b3d463e9bcf service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:02:24.453696 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 749/1513] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:02:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:02:25.293195 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73433701-a089-49ad-94e9-fa3a82979d61 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:02:25.293772 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73433701-a089-49ad-94e9-fa3a82979d61 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:02:25.293907 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73433701-a089-49ad-94e9-fa3a82979d61 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:02:25.295269 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 765/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 16:02:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 16:02:25.300013 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c176dc9-f550-4bfb-b14d-f7f80b5e69ed None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:02:25.300452 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c176dc9-f550-4bfb-b14d-f7f80b5e69ed None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:02:25.300587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c176dc9-f550-4bfb-b14d-f7f80b5e69ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:02:25.319568 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4c176dc9-f550-4bfb-b14d-f7f80b5e69ed None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:02:25.320764 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4c176dc9-f550-4bfb-b14d-f7f80b5e69ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:02:25.329174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 750/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 16:02:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:02:25.482346 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73433701-a089-49ad-94e9-fa3a82979d61 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:02:25.497253 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73433701-a089-49ad-94e9-fa3a82979d61 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:02:25.502726 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c58e91-de0f-435d-8f34-fd6cab19d847 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:02:25.503358 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c58e91-de0f-435d-8f34-fd6cab19d847 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:02:25.503642 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c58e91-de0f-435d-8f34-fd6cab19d847 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:02:25.503919 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c58e91-de0f-435d-8f34-fd6cab19d847 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:02:25.512749 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c58e91-de0f-435d-8f34-fd6cab19d847 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:02:25.514381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c58e91-de0f-435d-8f34-fd6cab19d847 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:02:25.523665 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 766/1516] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:02:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:02:26.599755 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bafcf3-bad7-4c4b-8f7b-af1303b8b82e None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:02:26.600308 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bafcf3-bad7-4c4b-8f7b-af1303b8b82e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:02:26.600308 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bafcf3-bad7-4c4b-8f7b-af1303b8b82e None None] PATH_INFO: `/v3` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:02:26.601626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 751/1517] 10.222.0.66 () {54 vars in 925 bytes} [Thu Jul 21 16:02:26 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:02:26.608089 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb344e-a8ae-4c03-8142-103896c9fded None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:02:26.608511 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb344e-a8ae-4c03-8142-103896c9fded None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:02:26.608628 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb344e-a8ae-4c03-8142-103896c9fded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:02:26.663930 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-16bb344e-a8ae-4c03-8142-103896c9fded None None] MFA Rules not processed for user `822d9d3521db4a9688f51c6313c7dd74`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:02:26.665127 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-16bb344e-a8ae-4c03-8142-103896c9fded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:02:26.682329 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:02:26.736236 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 767/1518] 10.222.0.66 () {58 vars in 1015 bytes} [Thu Jul 21 16:02:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:02:30.668381 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466528c-126e-478e-8a55-4d0ef0b9c1f5 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:02:30.668916 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466528c-126e-478e-8a55-4d0ef0b9c1f5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:02:30.668916 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466528c-126e-478e-8a55-4d0ef0b9c1f5 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:02:30.670336 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 752/1519] 10.222.0.52 () {54 vars in 961 bytes} [Thu Jul 21 16:02:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 16:02:30.675478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49b5a0-0810-4c41-8f20-6b3c3aee823c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:02:30.675927 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49b5a0-0810-4c41-8f20-6b3c3aee823c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:02:30.676037 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49b5a0-0810-4c41-8f20-6b3c3aee823c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:02:30.695488 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5e49b5a0-0810-4c41-8f20-6b3c3aee823c None None] MFA Rules not processed for user `822d9d3521db4a9688f51c6313c7dd74`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:02:30.697304 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5e49b5a0-0810-4c41-8f20-6b3c3aee823c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:02:30.706022 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 768/1520] 10.222.0.52 () {58 vars in 1058 bytes} [Thu Jul 21 16:02:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:02:30.713976 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466528c-126e-478e-8a55-4d0ef0b9c1f5 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:02:30.733063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466528c-126e-478e-8a55-4d0ef0b9c1f5 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:02:30.738536 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e19495-2dbc-4a80-84b4-0d12e59db7ae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822d9d3521db4a9688f51c6313c7dd74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:02:30.739403 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e19495-2dbc-4a80-84b4-0d12e59db7ae service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:02:30.739687 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e19495-2dbc-4a80-84b4-0d12e59db7ae service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:02:30.739964 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e19495-2dbc-4a80-84b4-0d12e59db7ae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:02:30.749045 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e19495-2dbc-4a80-84b4-0d12e59db7ae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:02:30.750661 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e19495-2dbc-4a80-84b4-0d12e59db7ae service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:02:30.761100 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 753/1521] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:02:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:02:36.752828 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69506a52-638e-44f5-aeea-9d606164f47f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:02:36.753405 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69506a52-638e-44f5-aeea-9d606164f47f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:02:36.753405 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69506a52-638e-44f5-aeea-9d606164f47f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:02:36.754769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 769/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 16:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 16:02:36.760461 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af781216-7a6f-4848-aa4d-5aa439856371 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:02:36.760871 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af781216-7a6f-4848-aa4d-5aa439856371 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:02:36.761018 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af781216-7a6f-4848-aa4d-5aa439856371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:02:36.778620 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-af781216-7a6f-4848-aa4d-5aa439856371 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:02:36.779822 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-af781216-7a6f-4848-aa4d-5aa439856371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:02:36.787595 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 754/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 16:02:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:02:36.940753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69506a52-638e-44f5-aeea-9d606164f47f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:02:36.955470 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69506a52-638e-44f5-aeea-9d606164f47f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:02:36.960802 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d68c8-b340-43e7-a156-28468035fcda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:02:36.961437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d68c8-b340-43e7-a156-28468035fcda service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:02:36.961811 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d68c8-b340-43e7-a156-28468035fcda service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:02:36.962613 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d68c8-b340-43e7-a156-28468035fcda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:02:36.971568 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d68c8-b340-43e7-a156-28468035fcda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:02:36.973437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d68c8-b340-43e7-a156-28468035fcda service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:02:36.982970 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 770/1524] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:02:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:02:37.814048 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ca42e-c57d-4f3e-ab8c-ed90daab44ba None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:02:37.814593 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ca42e-c57d-4f3e-ab8c-ed90daab44ba None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:02:37.814724 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ca42e-c57d-4f3e-ab8c-ed90daab44ba None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:02:37.816047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 755/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 16:02:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 16:02:37.820729 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecaedd-cb78-47f0-aab7-1b3a2f18f2c1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:02:37.821137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecaedd-cb78-47f0-aab7-1b3a2f18f2c1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:02:37.821310 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecaedd-cb78-47f0-aab7-1b3a2f18f2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:02:37.838847 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-caecaedd-cb78-47f0-aab7-1b3a2f18f2c1 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:02:37.840029 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-caecaedd-cb78-47f0-aab7-1b3a2f18f2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:02:37.847699 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 771/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 16:02:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:02:37.998270 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ca42e-c57d-4f3e-ab8c-ed90daab44ba None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:02:38.020163 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ca42e-c57d-4f3e-ab8c-ed90daab44ba None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:02:38.026635 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea4965c-6278-4a04-9035-b4e2af5eaf50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:02:38.027287 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea4965c-6278-4a04-9035-b4e2af5eaf50 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:02:38.027571 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea4965c-6278-4a04-9035-b4e2af5eaf50 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:02:38.027850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea4965c-6278-4a04-9035-b4e2af5eaf50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:02:38.036267 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea4965c-6278-4a04-9035-b4e2af5eaf50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:02:38.037885 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea4965c-6278-4a04-9035-b4e2af5eaf50 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:02:38.047234 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 756/1527] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:02:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:02:42.048674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93ab69-b642-4485-aef2-f0a2fcaba4b2 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:02:42.049256 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93ab69-b642-4485-aef2-f0a2fcaba4b2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:02:42.049256 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93ab69-b642-4485-aef2-f0a2fcaba4b2 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:02:42.050917 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 772/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 16:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 16:02:42.056658 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9f055-9494-45ab-b0f5-3484aeda9a59 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:02:42.057055 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9f055-9494-45ab-b0f5-3484aeda9a59 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:02:42.057206 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9f055-9494-45ab-b0f5-3484aeda9a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:02:42.075165 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c8a9f055-9494-45ab-b0f5-3484aeda9a59 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:02:42.076358 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c8a9f055-9494-45ab-b0f5-3484aeda9a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:02:42.084055 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 757/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 16:02:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:02:42.156129 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93ab69-b642-4485-aef2-f0a2fcaba4b2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:02:42.157885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2a93ab69-b642-4485-aef2-f0a2fcaba4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:02:42.236486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93ab69-b642-4485-aef2-f0a2fcaba4b2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:02:42.240734 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a7d67-a3b3-4f1a-8817-2606384b40f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:02:42.241396 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a7d67-a3b3-4f1a-8817-2606384b40f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:02:42.241714 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a7d67-a3b3-4f1a-8817-2606384b40f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:02:42.241994 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a7d67-a3b3-4f1a-8817-2606384b40f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:02:42.251933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9a7d67-a3b3-4f1a-8817-2606384b40f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:02:42.253370 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9a7d67-a3b3-4f1a-8817-2606384b40f3 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:02:42.262852 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 773/1530] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:02:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:02:44.274855 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09522b9c-0c5e-4ae6-9433-6531e19adb90 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:02:44.275246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09522b9c-0c5e-4ae6-9433-6531e19adb90 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:02:44.275505 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09522b9c-0c5e-4ae6-9433-6531e19adb90 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:02:44.276889 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 758/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 16:02:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 16:02:44.282311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ec150-f159-46e5-851e-960c7da5479f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:02:44.282721 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ec150-f159-46e5-851e-960c7da5479f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:02:44.282816 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ec150-f159-46e5-851e-960c7da5479f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:02:44.300354 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-644ec150-f159-46e5-851e-960c7da5479f None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:02:44.301542 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-644ec150-f159-46e5-851e-960c7da5479f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:02:44.309960 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 774/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 16:02:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:02:44.345730 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09522b9c-0c5e-4ae6-9433-6531e19adb90 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:02:44.357217 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09522b9c-0c5e-4ae6-9433-6531e19adb90 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:02:44.360863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ee2834-abae-4581-a92d-9cc009170f82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:02:44.361493 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee2834-abae-4581-a92d-9cc009170f82 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:02:44.361817 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee2834-abae-4581-a92d-9cc009170f82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:02:44.362097 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee2834-abae-4581-a92d-9cc009170f82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:02:44.371525 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee2834-abae-4581-a92d-9cc009170f82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:02:44.372946 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee2834-abae-4581-a92d-9cc009170f82 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:02:44.382265 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 759/1533] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:02:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:02:48.064258 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0bba91-6daf-40ba-a2c6-91a40fff0574 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:02:48.064799 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0bba91-6daf-40ba-a2c6-91a40fff0574 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:02:48.064799 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0bba91-6daf-40ba-a2c6-91a40fff0574 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:02:48.066222 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 775/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 16:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 16:02:48.071939 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40827d0-12ac-41fc-800c-716f97ae5e39 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:02:48.072364 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40827d0-12ac-41fc-800c-716f97ae5e39 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:02:48.072496 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40827d0-12ac-41fc-800c-716f97ae5e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:02:48.090609 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d40827d0-12ac-41fc-800c-716f97ae5e39 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:02:48.091805 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d40827d0-12ac-41fc-800c-716f97ae5e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:02:48.099662 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 760/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 16:02:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:02:48.400060 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0bba91-6daf-40ba-a2c6-91a40fff0574 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:02:48.401749 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0b0bba91-6daf-40ba-a2c6-91a40fff0574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:02:48.465177 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:02:48.478765 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0bba91-6daf-40ba-a2c6-91a40fff0574 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:02:48.482281 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa375d-d7b6-4a1c-84b4-93bbd44e493a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:02:48.482915 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa375d-d7b6-4a1c-84b4-93bbd44e493a service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:02:48.483193 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa375d-d7b6-4a1c-84b4-93bbd44e493a service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:02:48.483464 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa375d-d7b6-4a1c-84b4-93bbd44e493a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:02:48.492194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afa375d-d7b6-4a1c-84b4-93bbd44e493a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:02:48.494152 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afa375d-d7b6-4a1c-84b4-93bbd44e493a service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:02:48.503309 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 776/1536] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:02:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:02:50.552871 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a48bc08-f927-439e-9e44-0ca887aa071e None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:02:50.553404 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a48bc08-f927-439e-9e44-0ca887aa071e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:02:50.553404 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a48bc08-f927-439e-9e44-0ca887aa071e None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:02:50.554754 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 761/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 16:02:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 16:02:50.559912 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0dbea-f9bc-4f5e-8c95-aa990a47d200 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:02:50.560339 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0dbea-f9bc-4f5e-8c95-aa990a47d200 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:02:50.560457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0dbea-f9bc-4f5e-8c95-aa990a47d200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:02:50.578621 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8cd0dbea-f9bc-4f5e-8c95-aa990a47d200 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:02:50.579885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8cd0dbea-f9bc-4f5e-8c95-aa990a47d200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:02:50.588638 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 777/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 16:02:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:02:50.888533 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a48bc08-f927-439e-9e44-0ca887aa071e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:02:50.890239 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2a48bc08-f927-439e-9e44-0ca887aa071e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:02:50.899305 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=84 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:02:50.905404 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a48bc08-f927-439e-9e44-0ca887aa071e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:02:50.908935 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c74726-e306-401c-a22d-0ae9570961fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:02:50.909588 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c74726-e306-401c-a22d-0ae9570961fd service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:02:50.909926 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c74726-e306-401c-a22d-0ae9570961fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:02:50.910194 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c74726-e306-401c-a22d-0ae9570961fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:02:50.919679 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c74726-e306-401c-a22d-0ae9570961fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:02:50.921113 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c74726-e306-401c-a22d-0ae9570961fd service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:02:50.930435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 762/1539] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:02:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:03:00.573966 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=11 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:03:00.926408 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:21.710034 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d66a52-e41f-433a-a7bb-2d754969aa00 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:21.710677 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d66a52-e41f-433a-a7bb-2d754969aa00 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:21.710677 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d66a52-e41f-433a-a7bb-2d754969aa00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:21.734984 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e3d66a52-e41f-433a-a7bb-2d754969aa00 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:21.736221 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e3d66a52-e41f-433a-a7bb-2d754969aa00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:21.749375 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 778/1540] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:21.757869 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f472663-1faf-4663-b15c-2f265cc15ff0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:21.758465 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f472663-1faf-4663-b15c-2f265cc15ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:21.758465 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f472663-1faf-4663-b15c-2f265cc15ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:21.794632 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7f472663-1faf-4663-b15c-2f265cc15ff0 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:21.795897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7f472663-1faf-4663-b15c-2f265cc15ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:21.807921 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 763/1541] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:21.813328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d66a52-e41f-433a-a7bb-2d754969aa00 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:21.828490 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d66a52-e41f-433a-a7bb-2d754969aa00 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:21.839114 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629225d-9feb-40e2-ac18-f44cb4ad4aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:21.839819 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629225d-9feb-40e2-ac18-f44cb4ad4aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:21.840101 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629225d-9feb-40e2-ac18-f44cb4ad4aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:21.840375 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629225d-9feb-40e2-ac18-f44cb4ad4aa4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:21.841132 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1629225d-9feb-40e2-ac18-f44cb4ad4aa4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:04:21.841442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1629225d-9feb-40e2-ac18-f44cb4ad4aa4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:21.843205 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1629225d-9feb-40e2-ac18-f44cb4ad4aa4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:21.862086 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 779/1542] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:04:21.870992 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f472663-1faf-4663-b15c-2f265cc15ff0 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:21.885953 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f472663-1faf-4663-b15c-2f265cc15ff0 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:21.894341 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326b6e6-30cd-4350-989c-d925c39c4c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:21.895568 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326b6e6-30cd-4350-989c-d925c39c4c54 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:21.895864 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326b6e6-30cd-4350-989c-d925c39c4c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:21.896146 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326b6e6-30cd-4350-989c-d925c39c4c54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:21.896871 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f326b6e6-30cd-4350-989c-d925c39c4c54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:21.899172 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f326b6e6-30cd-4350-989c-d925c39c4c54 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:21.960404 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 764/1543] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:04:21 2022] POST /identity/v3/projects => generated 373 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:21.966227 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629225d-9feb-40e2-ac18-f44cb4ad4aa4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:21.978961 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629225d-9feb-40e2-ac18-f44cb4ad4aa4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:22.057922 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef7a0ca-3103-4d8b-9fce-0806ddbbe21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:22.058604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7a0ca-3103-4d8b-9fce-0806ddbbe21e admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:22.058890 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7a0ca-3103-4d8b-9fce-0806ddbbe21e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:22.059169 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7a0ca-3103-4d8b-9fce-0806ddbbe21e admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:22.059866 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef7a0ca-3103-4d8b-9fce-0806ddbbe21e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:22.061833 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef7a0ca-3103-4d8b-9fce-0806ddbbe21e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:22.081342 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1199922-d31e-4950-9a1c-3afa5679b159 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:22.081739 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1199922-d31e-4950-9a1c-3afa5679b159 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:22.081920 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1199922-d31e-4950-9a1c-3afa5679b159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:22.088769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 780/1544] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:04:21 2022] POST /identity/v3/users => generated 287 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:22.093279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef7a0ca-3103-4d8b-9fce-0806ddbbe21e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:22.103669 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef7a0ca-3103-4d8b-9fce-0806ddbbe21e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:22.113430 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9c2ea-4f40-40a5-b626-19ab208e4ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:22.114146 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9c2ea-4f40-40a5-b626-19ab208e4ace admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:22.114432 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9c2ea-4f40-40a5-b626-19ab208e4ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:22.114710 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9c2ea-4f40-40a5-b626-19ab208e4ace admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:22.115546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d9c2ea-4f40-40a5-b626-19ab208e4ace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:22.117167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d9c2ea-4f40-40a5-b626-19ab208e4ace admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:22.121372 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a1199922-d31e-4950-9a1c-3afa5679b159 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:22.122750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a1199922-d31e-4950-9a1c-3afa5679b159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:22.133274 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 781/1545] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:22.137816 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9c2ea-4f40-40a5-b626-19ab208e4ace admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:22.142474 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 765/1546] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:22.148760 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9c2ea-4f40-40a5-b626-19ab208e4ace admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:22.152509 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c3307-4208-401b-a635-0b1a3a2fe76a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:22.152803 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c3307-4208-401b-a635-0b1a3a2fe76a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:22.153088 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c3307-4208-401b-a635-0b1a3a2fe76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:22.157491 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4a0f7-e5dc-4ec0-ba32-d69fbb284816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:22.158199 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4a0f7-e5dc-4ec0-ba32-d69fbb284816 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:22.158572 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4a0f7-e5dc-4ec0-ba32-d69fbb284816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:22.158858 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4a0f7-e5dc-4ec0-ba32-d69fbb284816 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:22.159589 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c4a0f7-e5dc-4ec0-ba32-d69fbb284816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:22.161215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c4a0f7-e5dc-4ec0-ba32-d69fbb284816 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:22.177379 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 782/1547] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 16:04:22.182156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4a0f7-e5dc-4ec0-ba32-d69fbb284816 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:22.192296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4a0f7-e5dc-4ec0-ba32-d69fbb284816 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:22.198596 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3b7c3307-4208-401b-a635-0b1a3a2fe76a None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:22.199832 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3b7c3307-4208-401b-a635-0b1a3a2fe76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:22.201917 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0ac74-9bd0-437b-999c-3bd548785040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:22.202828 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0ac74-9bd0-437b-999c-3bd548785040 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:22.203208 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0ac74-9bd0-437b-999c-3bd548785040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:22.203495 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0ac74-9bd0-437b-999c-3bd548785040 admin admin] PATH_INFO: `/v3/projects/60f9ebe8843840599f6a084de9539156/users/87d7fe7d695c4dffbfeffc2edfa1c842/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:22.224691 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f0ac74-9bd0-437b-999c-3bd548785040 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60f9ebe8843840599f6a084de9539156, user_id=87d7fe7d695c4dffbfeffc2edfa1c842, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:22.225002 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 766/1548] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:04:22.225706 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-13f0ac74-9bd0-437b-999c-3bd548785040 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 16:04:22.225706 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: warnings.warn(msg) Jul 21 16:04:22.225706 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: Jul 21 16:04:22.227639 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f0ac74-9bd0-437b-999c-3bd548785040 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:22.230365 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c3307-4208-401b-a635-0b1a3a2fe76a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:22.249639 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c3307-4208-401b-a635-0b1a3a2fe76a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:22.251540 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 783/1549] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:04:22 2022] PUT /identity/v3/projects/60f9ebe8843840599f6a084de9539156/users/87d7fe7d695c4dffbfeffc2edfa1c842/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:22.260974 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9aeb00-e501-4d9e-a7f4-7159fdc1c0f5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:22.261266 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9aeb00-e501-4d9e-a7f4-7159fdc1c0f5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:22.261551 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9aeb00-e501-4d9e-a7f4-7159fdc1c0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:22.295563 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-bd9aeb00-e501-4d9e-a7f4-7159fdc1c0f5 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:22.296813 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bd9aeb00-e501-4d9e-a7f4-7159fdc1c0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:22.371793 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9388a32-1508-4dd4-b497-7777d99c880a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:22.372879 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9388a32-1508-4dd4-b497-7777d99c880a admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:22.373380 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9388a32-1508-4dd4-b497-7777d99c880a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:22.373911 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9388a32-1508-4dd4-b497-7777d99c880a admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:22.374849 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9388a32-1508-4dd4-b497-7777d99c880a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:04:22.375391 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9388a32-1508-4dd4-b497-7777d99c880a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:22.376625 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: WARNING py.warnings [None req-a9388a32-1508-4dd4-b497-7777d99c880a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 16:04:22.376625 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: warnings.warn(msg) Jul 21 16:04:22.376625 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: Jul 21 16:04:22.384632 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9388a32-1508-4dd4-b497-7777d99c880a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:22.401605 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 767/1550] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:04:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 16:04:22.403274 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d3006c-4f84-41e3-89ab-06538bf74546 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:22.403792 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d3006c-4f84-41e3-89ab-06538bf74546 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:22.404287 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d3006c-4f84-41e3-89ab-06538bf74546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:22.416488 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 784/1551] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:04:22.417910 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2f9b6-d194-44cb-87c1-fe6acad8695a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:22.418216 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2f9b6-d194-44cb-87c1-fe6acad8695a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:22.418478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2f9b6-d194-44cb-87c1-fe6acad8695a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:22.438339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e2d3006c-4f84-41e3-89ab-06538bf74546 None None] MFA Rules not processed for user `87d7fe7d695c4dffbfeffc2edfa1c842`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:22.439556 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e2d3006c-4f84-41e3-89ab-06538bf74546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:22.441254 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-93b2f9b6-d194-44cb-87c1-fe6acad8695a None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:22.442455 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-93b2f9b6-d194-44cb-87c1-fe6acad8695a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:22.451165 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 785/1552] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:22.452376 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536682a-4f4f-4fc8-9c3d-488786c43591 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:22.452660 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536682a-4f4f-4fc8-9c3d-488786c43591 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:22.453118 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536682a-4f4f-4fc8-9c3d-488786c43591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:22.471440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3536682a-4f4f-4fc8-9c3d-488786c43591 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:22.472627 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3536682a-4f4f-4fc8-9c3d-488786c43591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:22.482003 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 786/1553] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:22.483231 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e744b15-0ba5-4f19-920e-e923d2becf7f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:22.483522 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e744b15-0ba5-4f19-920e-e923d2becf7f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:22.483800 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e744b15-0ba5-4f19-920e-e923d2becf7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:22.501735 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2e744b15-0ba5-4f19-920e-e923d2becf7f None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:22.502915 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2e744b15-0ba5-4f19-920e-e923d2becf7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:22.511412 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 787/1554] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:22.512598 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fa0b8-83bb-4b0e-a004-a798d0c43ed8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:22.512885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fa0b8-83bb-4b0e-a004-a798d0c43ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:22.513156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fa0b8-83bb-4b0e-a004-a798d0c43ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:22.519736 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 768/1555] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:22.520231 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:22.520620 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d3006c-4f84-41e3-89ab-06538bf74546 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:22.531999 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d3006c-4f84-41e3-89ab-06538bf74546 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:22.534090 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-976fa0b8-83bb-4b0e-a004-a798d0c43ed8 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:22.535444 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-976fa0b8-83bb-4b0e-a004-a798d0c43ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:22.536646 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04f9f0-2939-479a-a2fc-74a9da404969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:22.537298 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04f9f0-2939-479a-a2fc-74a9da404969 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:22.537595 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04f9f0-2939-479a-a2fc-74a9da404969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:22.537901 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04f9f0-2939-479a-a2fc-74a9da404969 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:22.538624 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f04f9f0-2939-479a-a2fc-74a9da404969 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:22.540547 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f04f9f0-2939-479a-a2fc-74a9da404969 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:22.543682 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 788/1556] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:22.544961 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b5b330-92d3-4026-ad2b-45be19cf5f00 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:22.545251 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b5b330-92d3-4026-ad2b-45be19cf5f00 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:22.545532 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b5b330-92d3-4026-ad2b-45be19cf5f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:22.563937 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b1b5b330-92d3-4026-ad2b-45be19cf5f00 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:22.565206 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b1b5b330-92d3-4026-ad2b-45be19cf5f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:22.575623 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 769/1557] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:04:22 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:22.575832 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 789/1558] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:22.575932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af7b5f-0483-4e5a-941c-08e64819a6b0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:22.575932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af7b5f-0483-4e5a-941c-08e64819a6b0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:22.575932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af7b5f-0483-4e5a-941c-08e64819a6b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:22.576162 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b98025-a06b-4d0b-b370-09c654d92a1d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:22.576162 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b98025-a06b-4d0b-b370-09c654d92a1d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:22.576162 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b98025-a06b-4d0b-b370-09c654d92a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:22.593660 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d1b98025-a06b-4d0b-b370-09c654d92a1d None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:22.594281 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-05af7b5f-0483-4e5a-941c-08e64819a6b0 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:22.594877 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d1b98025-a06b-4d0b-b370-09c654d92a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:22.595462 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-05af7b5f-0483-4e5a-941c-08e64819a6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:22.600991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:22.654920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 790/1559] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:22.656289 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9a037-559f-470f-893c-6007301671b9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:22.656581 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9a037-559f-470f-893c-6007301671b9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:22.656858 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9a037-559f-470f-893c-6007301671b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:22.661847 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 770/1560] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:22.663559 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4a5d5-cbce-431a-8d20-8490f0c4c330 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:22.663926 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4a5d5-cbce-431a-8d20-8490f0c4c330 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:22.664214 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4a5d5-cbce-431a-8d20-8490f0c4c330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:22.675201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ccb9a037-559f-470f-893c-6007301671b9 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:22.676443 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ccb9a037-559f-470f-893c-6007301671b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:22.683275 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-04b4a5d5-cbce-431a-8d20-8490f0c4c330 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:22.684281 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 791/1561] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:22.687974 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-04b4a5d5-cbce-431a-8d20-8490f0c4c330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:22.688186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb9a037-559f-470f-893c-6007301671b9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:22.695955 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb9a037-559f-470f-893c-6007301671b9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:22.697233 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 771/1562] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:22.698724 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51ca35-1ae7-4841-be3f-cffbfde315e0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:22.699082 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51ca35-1ae7-4841-be3f-cffbfde315e0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:22.699536 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51ca35-1ae7-4841-be3f-cffbfde315e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:22.703672 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4d582-24be-4dde-8c26-6a0f16acffc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:22.704311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4d582-24be-4dde-8c26-6a0f16acffc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:22.704596 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4d582-24be-4dde-8c26-6a0f16acffc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:22.704869 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4d582-24be-4dde-8c26-6a0f16acffc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:22.705550 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e4d582-24be-4dde-8c26-6a0f16acffc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:22.706607 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-40e4d582-24be-4dde-8c26-6a0f16acffc8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 16:04:22.706607 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: warnings.warn(msg) Jul 21 16:04:22.706607 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: Jul 21 16:04:22.707790 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e4d582-24be-4dde-8c26-6a0f16acffc8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:22.717849 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ef51ca35-1ae7-4841-be3f-cffbfde315e0 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:22.719051 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ef51ca35-1ae7-4841-be3f-cffbfde315e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:22.727505 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 772/1563] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:22.731262 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51ca35-1ae7-4841-be3f-cffbfde315e0 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:22.735444 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 792/1564] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:04:22 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:22.736103 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4d582-24be-4dde-8c26-6a0f16acffc8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:22.738518 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51ca35-1ae7-4841-be3f-cffbfde315e0 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:22.797294 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca4954-050a-4d5f-8c63-9aa4933a5628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:22.798085 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca4954-050a-4d5f-8c63-9aa4933a5628 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:22.798300 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca4954-050a-4d5f-8c63-9aa4933a5628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:22.798583 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca4954-050a-4d5f-8c63-9aa4933a5628 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:22.799286 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca4954-050a-4d5f-8c63-9aa4933a5628 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:22.800324 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: WARNING py.warnings [None req-29ca4954-050a-4d5f-8c63-9aa4933a5628 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 16:04:22.800324 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: warnings.warn(msg) Jul 21 16:04:22.800324 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: Jul 21 16:04:22.801524 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca4954-050a-4d5f-8c63-9aa4933a5628 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:22.802833 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4d582-24be-4dde-8c26-6a0f16acffc8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:22.808676 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2052b8a1-180e-4ec2-94bf-a2dec5b7f5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:22.809808 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2052b8a1-180e-4ec2-94bf-a2dec5b7f5bf admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:22.810406 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2052b8a1-180e-4ec2-94bf-a2dec5b7f5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:22.810713 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2052b8a1-180e-4ec2-94bf-a2dec5b7f5bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:22.811779 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2052b8a1-180e-4ec2-94bf-a2dec5b7f5bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:04:22.812104 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2052b8a1-180e-4ec2-94bf-a2dec5b7f5bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:22.814085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2052b8a1-180e-4ec2-94bf-a2dec5b7f5bf admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:22.825670 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 773/1565] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:04:22 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:22.825670 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca4954-050a-4d5f-8c63-9aa4933a5628 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:22.829698 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 793/1566] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:04:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:04:22.830989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec222f-1299-48ae-9639-11633791f734 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:22.831384 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec222f-1299-48ae-9639-11633791f734 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:22.831564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec222f-1299-48ae-9639-11633791f734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:22.834144 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca4954-050a-4d5f-8c63-9aa4933a5628 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:22.839703 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70028f77-f9ab-4791-a3cc-41e9482ad9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:22.840356 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70028f77-f9ab-4791-a3cc-41e9482ad9ff admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:22.840647 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70028f77-f9ab-4791-a3cc-41e9482ad9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:22.840928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70028f77-f9ab-4791-a3cc-41e9482ad9ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:22.841724 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70028f77-f9ab-4791-a3cc-41e9482ad9ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:04:22.842040 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70028f77-f9ab-4791-a3cc-41e9482ad9ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:22.843643 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70028f77-f9ab-4791-a3cc-41e9482ad9ff admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:22.850305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-39ec222f-1299-48ae-9639-11633791f734 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:22.851505 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-39ec222f-1299-48ae-9639-11633791f734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:22.856113 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 774/1567] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:04:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:04:22.856975 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70028f77-f9ab-4791-a3cc-41e9482ad9ff admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:22.859635 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 794/1568] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:22.860262 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ec222f-1299-48ae-9639-11633791f734 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:22.866672 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70028f77-f9ab-4791-a3cc-41e9482ad9ff admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:22.871798 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4067a604-7b48-481e-8df8-970edab00bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:22.872160 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ec222f-1299-48ae-9639-11633791f734 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:22.872436 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067a604-7b48-481e-8df8-970edab00bf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:22.872772 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067a604-7b48-481e-8df8-970edab00bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:22.873002 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067a604-7b48-481e-8df8-970edab00bf9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:22.873755 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4067a604-7b48-481e-8df8-970edab00bf9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:04:22.874069 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4067a604-7b48-481e-8df8-970edab00bf9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:22.875698 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4067a604-7b48-481e-8df8-970edab00bf9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:22.876818 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847191d-b5b1-44c8-9832-c4b64c39628a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:22.877455 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847191d-b5b1-44c8-9832-c4b64c39628a admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:22.877766 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847191d-b5b1-44c8-9832-c4b64c39628a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:22.878044 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847191d-b5b1-44c8-9832-c4b64c39628a admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:22.878767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847191d-b5b1-44c8-9832-c4b64c39628a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:04:22.879073 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847191d-b5b1-44c8-9832-c4b64c39628a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:22.880655 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847191d-b5b1-44c8-9832-c4b64c39628a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:22.892687 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 775/1569] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:04:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:04:22.892687 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4067a604-7b48-481e-8df8-970edab00bf9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:22.892944 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 795/1570] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:04:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:04:22.893499 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847191d-b5b1-44c8-9832-c4b64c39628a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:22.906834 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847191d-b5b1-44c8-9832-c4b64c39628a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:22.907241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4067a604-7b48-481e-8df8-970edab00bf9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:22.911453 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1adabf-5fb8-4841-b53c-88d6ca7439d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:22.911800 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa9918-1f76-42e9-981d-1c1244d2cbb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:22.912102 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1adabf-5fb8-4841-b53c-88d6ca7439d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:22.912388 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1adabf-5fb8-4841-b53c-88d6ca7439d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:22.912497 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa9918-1f76-42e9-981d-1c1244d2cbb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:22.912669 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1adabf-5fb8-4841-b53c-88d6ca7439d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:22.912776 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa9918-1f76-42e9-981d-1c1244d2cbb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:22.913105 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa9918-1f76-42e9-981d-1c1244d2cbb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:22.913388 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1adabf-5fb8-4841-b53c-88d6ca7439d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:22.913933 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fa9918-1f76-42e9-981d-1c1244d2cbb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:04:22.914244 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fa9918-1f76-42e9-981d-1c1244d2cbb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:22.915367 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1adabf-5fb8-4841-b53c-88d6ca7439d6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:22.915877 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fa9918-1f76-42e9-981d-1c1244d2cbb3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:22.928702 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 776/1571] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:04:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:04:22.929676 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa9918-1f76-42e9-981d-1c1244d2cbb3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:22.939020 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 796/1572] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:04:22 2022] POST /identity/v3/users => generated 292 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:22.940369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1adabf-5fb8-4841-b53c-88d6ca7439d6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:22.949054 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa9918-1f76-42e9-981d-1c1244d2cbb3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:22.952098 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1adabf-5fb8-4841-b53c-88d6ca7439d6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:22.954030 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ba899-e7e4-481d-9706-02af4135552f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:22.954692 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ba899-e7e4-481d-9706-02af4135552f admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:22.954982 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ba899-e7e4-481d-9706-02af4135552f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:22.955265 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ba899-e7e4-481d-9706-02af4135552f admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:22.955993 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ba899-e7e4-481d-9706-02af4135552f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:22.957610 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ba899-e7e4-481d-9706-02af4135552f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:22.957749 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41e355-e884-409d-bfe4-9a9903854517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:22.958364 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41e355-e884-409d-bfe4-9a9903854517 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:22.958796 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41e355-e884-409d-bfe4-9a9903854517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:22.959239 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41e355-e884-409d-bfe4-9a9903854517 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:22.960376 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b41e355-e884-409d-bfe4-9a9903854517 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:22.963823 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b41e355-e884-409d-bfe4-9a9903854517 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:22.971746 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 777/1573] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:22.972753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ba899-e7e4-481d-9706-02af4135552f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:22.984494 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ba899-e7e4-481d-9706-02af4135552f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:22.985915 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:22.990643 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87897283-44d8-472b-bfc2-ed3eca11008a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:22.991366 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87897283-44d8-472b-bfc2-ed3eca11008a admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:22.991740 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87897283-44d8-472b-bfc2-ed3eca11008a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:22.992155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87897283-44d8-472b-bfc2-ed3eca11008a admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:22.992756 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87897283-44d8-472b-bfc2-ed3eca11008a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:22.998423 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 797/1574] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:04:22 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:22.998562 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87897283-44d8-472b-bfc2-ed3eca11008a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:22.998662 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41e355-e884-409d-bfe4-9a9903854517 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:23.027149 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 778/1575] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:04:22 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:23.027149 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87897283-44d8-472b-bfc2-ed3eca11008a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:23.044261 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:23.064971 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41e355-e884-409d-bfe4-9a9903854517 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:23.092847 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87897283-44d8-472b-bfc2-ed3eca11008a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:23.098733 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab45cc-0775-472a-a38a-5393a1c3002c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:23.099388 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab45cc-0775-472a-a38a-5393a1c3002c admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:23.099670 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab45cc-0775-472a-a38a-5393a1c3002c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:23.099950 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab45cc-0775-472a-a38a-5393a1c3002c admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:23.100638 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab45cc-0775-472a-a38a-5393a1c3002c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:23.102592 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab45cc-0775-472a-a38a-5393a1c3002c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:23.128685 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 779/1576] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:04:23 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:23.129620 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab45cc-0775-472a-a38a-5393a1c3002c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:23.132195 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af19dc-4122-48bf-aa2f-5b62d1dd039c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:23.132869 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af19dc-4122-48bf-aa2f-5b62d1dd039c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:23.133157 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af19dc-4122-48bf-aa2f-5b62d1dd039c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:23.133438 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af19dc-4122-48bf-aa2f-5b62d1dd039c admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:23.134830 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af19dc-4122-48bf-aa2f-5b62d1dd039c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:04:23.135145 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af19dc-4122-48bf-aa2f-5b62d1dd039c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:23.136764 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af19dc-4122-48bf-aa2f-5b62d1dd039c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:23.150149 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 798/1577] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:04:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:04:23.150940 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af19dc-4122-48bf-aa2f-5b62d1dd039c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:23.162421 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af19dc-4122-48bf-aa2f-5b62d1dd039c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:23.193229 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab45cc-0775-472a-a38a-5393a1c3002c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:23.199226 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ace712-897f-4d49-be54-ad16de6fa89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:23.199893 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ace712-897f-4d49-be54-ad16de6fa89a admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:23.200182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ace712-897f-4d49-be54-ad16de6fa89a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:23.200461 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ace712-897f-4d49-be54-ad16de6fa89a admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:23.201156 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ace712-897f-4d49-be54-ad16de6fa89a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:23.203100 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ace712-897f-4d49-be54-ad16de6fa89a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:23.220553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436ba36-a77c-470e-b04f-54d8a3d24412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:23.221290 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436ba36-a77c-470e-b04f-54d8a3d24412 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:23.221668 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436ba36-a77c-470e-b04f-54d8a3d24412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:23.221961 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436ba36-a77c-470e-b04f-54d8a3d24412 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:23.222659 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a436ba36-a77c-470e-b04f-54d8a3d24412 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:23.224896 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a436ba36-a77c-470e-b04f-54d8a3d24412 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:23.231830 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 780/1578] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:04:23 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:23.231830 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ace712-897f-4d49-be54-ad16de6fa89a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:23.253177 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 799/1579] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:04:23 2022] POST /identity/v3/projects => generated 367 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:23.254298 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436ba36-a77c-470e-b04f-54d8a3d24412 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:23.279967 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:23.298638 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ace712-897f-4d49-be54-ad16de6fa89a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:23.318674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436ba36-a77c-470e-b04f-54d8a3d24412 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:23.323758 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dbd00c-9032-4a7b-81c0-375e1ee7ce75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:23.324422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dbd00c-9032-4a7b-81c0-375e1ee7ce75 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:23.324706 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dbd00c-9032-4a7b-81c0-375e1ee7ce75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:23.324984 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dbd00c-9032-4a7b-81c0-375e1ee7ce75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:23.325741 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dbd00c-9032-4a7b-81c0-375e1ee7ce75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:23.327348 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dbd00c-9032-4a7b-81c0-375e1ee7ce75 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:23.330562 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:23.338340 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 800/1580] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:23.339143 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dbd00c-9032-4a7b-81c0-375e1ee7ce75 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:23.348979 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dbd00c-9032-4a7b-81c0-375e1ee7ce75 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:23.354654 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d5326-0d39-466f-b29d-79b72a789609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:23.355319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d5326-0d39-466f-b29d-79b72a789609 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:23.355930 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d5326-0d39-466f-b29d-79b72a789609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:23.356334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d5326-0d39-466f-b29d-79b72a789609 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:23.356914 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d5326-0d39-466f-b29d-79b72a789609 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:23.358875 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d5326-0d39-466f-b29d-79b72a789609 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:23.361316 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b4505-522c-4645-af0d-88326716e89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:23.362038 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b4505-522c-4645-af0d-88326716e89a admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:23.362319 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b4505-522c-4645-af0d-88326716e89a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:23.362595 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b4505-522c-4645-af0d-88326716e89a admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:23.363335 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462b4505-522c-4645-af0d-88326716e89a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:23.364955 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462b4505-522c-4645-af0d-88326716e89a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:23.375917 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 781/1581] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:23.376730 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b4505-522c-4645-af0d-88326716e89a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:23.387096 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b4505-522c-4645-af0d-88326716e89a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:23.391957 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 801/1582] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:04:23 2022] POST /identity/v3/users => generated 291 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:23.391957 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d5326-0d39-466f-b29d-79b72a789609 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:23.394080 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ddd19-c87b-4f57-b7eb-d68ef6bad8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:23.395048 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ddd19-c87b-4f57-b7eb-d68ef6bad8c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:23.395339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ddd19-c87b-4f57-b7eb-d68ef6bad8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:23.395627 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ddd19-c87b-4f57-b7eb-d68ef6bad8c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:23.396872 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9ddd19-c87b-4f57-b7eb-d68ef6bad8c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:23.399005 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9ddd19-c87b-4f57-b7eb-d68ef6bad8c4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:23.399574 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d5326-0d39-466f-b29d-79b72a789609 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:23.404669 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c2406-bae6-4634-82d0-907dd3bdd82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:23.405310 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c2406-bae6-4634-82d0-907dd3bdd82d admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:23.405619 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c2406-bae6-4634-82d0-907dd3bdd82d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:23.405902 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c2406-bae6-4634-82d0-907dd3bdd82d admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:23.406575 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3c2406-bae6-4634-82d0-907dd3bdd82d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:23.408486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3c2406-bae6-4634-82d0-907dd3bdd82d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:23.421134 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 782/1583] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:04:23 2022] POST /identity/v3/users => generated 287 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:23.422092 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ddd19-c87b-4f57-b7eb-d68ef6bad8c4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:23.430519 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 802/1584] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:04:23 2022] POST /identity/v3/users => generated 293 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:23.431535 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c2406-bae6-4634-82d0-907dd3bdd82d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:23.431754 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ddd19-c87b-4f57-b7eb-d68ef6bad8c4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:23.437477 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7976f8f3-7776-4e67-ab90-7abfeeab3539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:23.438153 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7976f8f3-7776-4e67-ab90-7abfeeab3539 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:23.438442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7976f8f3-7776-4e67-ab90-7abfeeab3539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:23.438848 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7976f8f3-7776-4e67-ab90-7abfeeab3539 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:23.439540 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7976f8f3-7776-4e67-ab90-7abfeeab3539 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:23.441026 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c2406-bae6-4634-82d0-907dd3bdd82d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:23.441431 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7976f8f3-7776-4e67-ab90-7abfeeab3539 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:23.445666 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20425996-365a-4fbc-85ee-f66d4eac1c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:23.446301 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20425996-365a-4fbc-85ee-f66d4eac1c6e admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:23.447109 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20425996-365a-4fbc-85ee-f66d4eac1c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:23.447388 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20425996-365a-4fbc-85ee-f66d4eac1c6e admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:23.448058 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20425996-365a-4fbc-85ee-f66d4eac1c6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:23.449980 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20425996-365a-4fbc-85ee-f66d4eac1c6e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:23.470329 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 783/1585] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:04:23 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:23.470980 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 803/1586] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:04:23 2022] POST /identity/v3/users => generated 294 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:23.474556 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20425996-365a-4fbc-85ee-f66d4eac1c6e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:23.474727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7976f8f3-7776-4e67-ab90-7abfeeab3539 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:23.484142 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20425996-365a-4fbc-85ee-f66d4eac1c6e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:23.535979 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7976f8f3-7776-4e67-ab90-7abfeeab3539 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:23.539915 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e7db25-2835-4aff-98d8-749e34a7c0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:23.540753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3466b83e-1e9f-400e-be06-f795724fee40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:23.540878 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e7db25-2835-4aff-98d8-749e34a7c0ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:23.541121 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e7db25-2835-4aff-98d8-749e34a7c0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:23.541398 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e7db25-2835-4aff-98d8-749e34a7c0ba admin admin] PATH_INFO: `/v3/projects/2e279d6a3e6d4ba7b697449111d569c2/users/a50cbf92286c420f9bee6e05c33dc440/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:23.541499 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3466b83e-1e9f-400e-be06-f795724fee40 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:23.541729 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3466b83e-1e9f-400e-be06-f795724fee40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:23.542014 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3466b83e-1e9f-400e-be06-f795724fee40 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:23.542743 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3466b83e-1e9f-400e-be06-f795724fee40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:23.544679 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3466b83e-1e9f-400e-be06-f795724fee40 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:23.555451 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e7db25-2835-4aff-98d8-749e34a7c0ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e279d6a3e6d4ba7b697449111d569c2, user_id=a50cbf92286c420f9bee6e05c33dc440, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:23.558516 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e7db25-2835-4aff-98d8-749e34a7c0ba admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:23.564903 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 784/1587] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:04:23 2022] POST /identity/v3/users => generated 284 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:23.565876 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3466b83e-1e9f-400e-be06-f795724fee40 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:23.573340 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 804/1588] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:04:23 2022] PUT /identity/v3/projects/2e279d6a3e6d4ba7b697449111d569c2/users/a50cbf92286c420f9bee6e05c33dc440/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:23.574316 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e7db25-2835-4aff-98d8-749e34a7c0ba admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:23.576471 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3466b83e-1e9f-400e-be06-f795724fee40 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:23.625169 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:23.634639 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820913ae-2ac2-4370-9cfa-5a7202788695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:23.635537 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820913ae-2ac2-4370-9cfa-5a7202788695 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:23.635825 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820913ae-2ac2-4370-9cfa-5a7202788695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:23.636114 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820913ae-2ac2-4370-9cfa-5a7202788695 admin admin] PATH_INFO: `/v3/projects/af6be1fe5a1a4a6193888065bc580778/users/2dbe3a4500974e57b875b8c10a62d10e/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:23.638362 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e7db25-2835-4aff-98d8-749e34a7c0ba admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:23.643007 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb46ec3-d04f-46a0-8ec9-891a14e3ae7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:23.643666 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb46ec3-d04f-46a0-8ec9-891a14e3ae7c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:23.643948 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb46ec3-d04f-46a0-8ec9-891a14e3ae7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:23.644229 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb46ec3-d04f-46a0-8ec9-891a14e3ae7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:23.644956 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb46ec3-d04f-46a0-8ec9-891a14e3ae7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:23.646578 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb46ec3-d04f-46a0-8ec9-891a14e3ae7c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:23.650418 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820913ae-2ac2-4370-9cfa-5a7202788695 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af6be1fe5a1a4a6193888065bc580778, user_id=2dbe3a4500974e57b875b8c10a62d10e, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:23.653071 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820913ae-2ac2-4370-9cfa-5a7202788695 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:23.656390 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 805/1589] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:23.657179 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb46ec3-d04f-46a0-8ec9-891a14e3ae7c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:23.667540 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb46ec3-d04f-46a0-8ec9-891a14e3ae7c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:23.670354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 785/1590] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:04:23 2022] PUT /identity/v3/projects/af6be1fe5a1a4a6193888065bc580778/users/2dbe3a4500974e57b875b8c10a62d10e/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:23.673541 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820913ae-2ac2-4370-9cfa-5a7202788695 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:23.674864 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:23.729271 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e65a7f1-1bfe-41b8-af2f-d1ff2a889451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:23.729983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65a7f1-1bfe-41b8-af2f-d1ff2a889451 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:23.730272 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65a7f1-1bfe-41b8-af2f-d1ff2a889451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:23.730552 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65a7f1-1bfe-41b8-af2f-d1ff2a889451 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:23.731291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e65a7f1-1bfe-41b8-af2f-d1ff2a889451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:23.733001 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e65a7f1-1bfe-41b8-af2f-d1ff2a889451 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:23.735495 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820913ae-2ac2-4370-9cfa-5a7202788695 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:23.740099 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d574f009-c6e1-4c93-89d0-681f54e45b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:23.740758 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574f009-c6e1-4c93-89d0-681f54e45b15 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:23.741072 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574f009-c6e1-4c93-89d0-681f54e45b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:23.741351 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574f009-c6e1-4c93-89d0-681f54e45b15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:23.742106 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d574f009-c6e1-4c93-89d0-681f54e45b15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:23.743179 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 806/1591] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:23.744284 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d574f009-c6e1-4c93-89d0-681f54e45b15 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:23.744529 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e65a7f1-1bfe-41b8-af2f-d1ff2a889451 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:23.754110 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e65a7f1-1bfe-41b8-af2f-d1ff2a889451 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:23.755499 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 786/1592] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:23.758614 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d574f009-c6e1-4c93-89d0-681f54e45b15 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:23.759659 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab60ad-5dd1-403b-adcd-d854884c0b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:23.760304 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab60ad-5dd1-403b-adcd-d854884c0b6d admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:23.760592 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab60ad-5dd1-403b-adcd-d854884c0b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:23.760903 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab60ad-5dd1-403b-adcd-d854884c0b6d admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:23.761555 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab60ad-5dd1-403b-adcd-d854884c0b6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:23.763547 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab60ad-5dd1-403b-adcd-d854884c0b6d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:23.768824 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d574f009-c6e1-4c93-89d0-681f54e45b15 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:23.773451 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e20e23e-26bf-4e50-ad4c-11a9d5f26215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:23.774130 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20e23e-26bf-4e50-ad4c-11a9d5f26215 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:23.774417 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20e23e-26bf-4e50-ad4c-11a9d5f26215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:23.774822 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20e23e-26bf-4e50-ad4c-11a9d5f26215 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:23.775679 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e20e23e-26bf-4e50-ad4c-11a9d5f26215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:23.777278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e20e23e-26bf-4e50-ad4c-11a9d5f26215 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:23.783932 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 807/1593] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:04:23 2022] POST /identity/v3/users => generated 289 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:23.784838 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab60ad-5dd1-403b-adcd-d854884c0b6d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:23.790481 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 787/1594] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:23.790481 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52011f-c824-4432-b312-19669781bcb1 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:23.790481 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52011f-c824-4432-b312-19669781bcb1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:23.790481 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52011f-c824-4432-b312-19669781bcb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:23.795167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab60ad-5dd1-403b-adcd-d854884c0b6d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:23.800402 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca8343-180c-41df-9581-f273493c9f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:23.801158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca8343-180c-41df-9581-f273493c9f4f admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:23.801457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca8343-180c-41df-9581-f273493c9f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:23.801804 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca8343-180c-41df-9581-f273493c9f4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:23.802610 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ca8343-180c-41df-9581-f273493c9f4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:23.804201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ca8343-180c-41df-9581-f273493c9f4f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:23.807398 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4d52011f-c824-4432-b312-19669781bcb1 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:23.808632 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4d52011f-c824-4432-b312-19669781bcb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:23.815580 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 808/1595] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:23.819285 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 788/1596] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:23.819285 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52011f-c824-4432-b312-19669781bcb1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:23.820588 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca8343-180c-41df-9581-f273493c9f4f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:23.828786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52011f-c824-4432-b312-19669781bcb1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:23.831616 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca8343-180c-41df-9581-f273493c9f4f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:23.833381 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265b40a-216d-4333-bd10-899e4b75ea8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:23.834082 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265b40a-216d-4333-bd10-899e4b75ea8f admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:23.834372 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265b40a-216d-4333-bd10-899e4b75ea8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:23.834655 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265b40a-216d-4333-bd10-899e4b75ea8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:23.835391 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1265b40a-216d-4333-bd10-899e4b75ea8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:23.836386 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c7826f-f905-4505-b263-a265d7cf94ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:23.837017 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1265b40a-216d-4333-bd10-899e4b75ea8f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:23.837137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7826f-f905-4505-b263-a265d7cf94ce admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:23.837341 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7826f-f905-4505-b263-a265d7cf94ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:23.837714 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7826f-f905-4505-b263-a265d7cf94ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:23.838443 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c7826f-f905-4505-b263-a265d7cf94ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:23.840021 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c7826f-f905-4505-b263-a265d7cf94ce admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:23.847238 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 789/1597] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:23.848348 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265b40a-216d-4333-bd10-899e4b75ea8f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:23.850950 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 809/1598] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:23.851932 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c7826f-f905-4505-b263-a265d7cf94ce admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:23.861457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265b40a-216d-4333-bd10-899e4b75ea8f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:23.861617 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c7826f-f905-4505-b263-a265d7cf94ce admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:23.866215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98414d6b-e082-46ac-9ab5-d5bade495014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:23.866353 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2741442-47d9-469a-8437-9520e71f39a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:23.866850 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98414d6b-e082-46ac-9ab5-d5bade495014 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:23.866971 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2741442-47d9-469a-8437-9520e71f39a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:23.867137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98414d6b-e082-46ac-9ab5-d5bade495014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:23.867249 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2741442-47d9-469a-8437-9520e71f39a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:23.867475 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2741442-47d9-469a-8437-9520e71f39a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:23.867584 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98414d6b-e082-46ac-9ab5-d5bade495014 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:23.868194 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2741442-47d9-469a-8437-9520e71f39a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:23.868817 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98414d6b-e082-46ac-9ab5-d5bade495014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:23.870302 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2741442-47d9-469a-8437-9520e71f39a0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:23.870408 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98414d6b-e082-46ac-9ab5-d5bade495014 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:23.880156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 810/1599] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:23.880960 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98414d6b-e082-46ac-9ab5-d5bade495014 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:23.881248 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 790/1600] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:23.882334 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2741442-47d9-469a-8437-9520e71f39a0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:23.893275 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98414d6b-e082-46ac-9ab5-d5bade495014 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:23.893656 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2741442-47d9-469a-8437-9520e71f39a0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:23.898061 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0871ccc5-6f0a-4bdb-a26e-d7b70e62f334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:23.898219 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c196fbc-2494-4fa3-92ce-8b9624966c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:23.898704 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871ccc5-6f0a-4bdb-a26e-d7b70e62f334 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:23.898814 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c196fbc-2494-4fa3-92ce-8b9624966c86 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:23.898989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871ccc5-6f0a-4bdb-a26e-d7b70e62f334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:23.899086 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c196fbc-2494-4fa3-92ce-8b9624966c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:23.899264 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871ccc5-6f0a-4bdb-a26e-d7b70e62f334 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:23.899372 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c196fbc-2494-4fa3-92ce-8b9624966c86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:23.899984 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0871ccc5-6f0a-4bdb-a26e-d7b70e62f334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:23.900134 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c196fbc-2494-4fa3-92ce-8b9624966c86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:23.901554 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0871ccc5-6f0a-4bdb-a26e-d7b70e62f334 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:23.901708 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c196fbc-2494-4fa3-92ce-8b9624966c86 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:23.911979 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 811/1601] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:23.912120 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 791/1602] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:23.915768 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0871ccc5-6f0a-4bdb-a26e-d7b70e62f334 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:23.915902 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c196fbc-2494-4fa3-92ce-8b9624966c86 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:23.924205 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0871ccc5-6f0a-4bdb-a26e-d7b70e62f334 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:23.928750 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d16a7-516e-45f5-9cf7-ef0d0291d358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:23.929399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d16a7-516e-45f5-9cf7-ef0d0291d358 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:23.929719 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d16a7-516e-45f5-9cf7-ef0d0291d358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:23.930012 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d16a7-516e-45f5-9cf7-ef0d0291d358 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:23.930727 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d16a7-516e-45f5-9cf7-ef0d0291d358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:23.932806 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d16a7-516e-45f5-9cf7-ef0d0291d358 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:23.942521 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 812/1603] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:23.943315 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d16a7-516e-45f5-9cf7-ef0d0291d358 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:23.953767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d16a7-516e-45f5-9cf7-ef0d0291d358 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:23.959006 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7308d625-c4e5-4401-9b7e-09e75db31f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:23.960077 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7308d625-c4e5-4401-9b7e-09e75db31f06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:23.960359 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7308d625-c4e5-4401-9b7e-09e75db31f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:23.960637 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7308d625-c4e5-4401-9b7e-09e75db31f06 admin admin] PATH_INFO: `/v3/projects/af6be1fe5a1a4a6193888065bc580778/users/2dbe3a4500974e57b875b8c10a62d10e/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:23.962651 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7308d625-c4e5-4401-9b7e-09e75db31f06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af6be1fe5a1a4a6193888065bc580778, user_id=2dbe3a4500974e57b875b8c10a62d10e, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:23.965399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7308d625-c4e5-4401-9b7e-09e75db31f06 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:23.970583 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c196fbc-2494-4fa3-92ce-8b9624966c86 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:23.975983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 813/1604] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:04:23 2022] PUT /identity/v3/projects/af6be1fe5a1a4a6193888065bc580778/users/2dbe3a4500974e57b875b8c10a62d10e/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:23.976733 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7308d625-c4e5-4401-9b7e-09e75db31f06 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:24.021987 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0596b18-8bcd-4eff-8059-e7cc4645d2e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:24.022642 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0596b18-8bcd-4eff-8059-e7cc4645d2e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:24.022931 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0596b18-8bcd-4eff-8059-e7cc4645d2e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:24.023212 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0596b18-8bcd-4eff-8059-e7cc4645d2e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:24.035341 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:24.039625 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7308d625-c4e5-4401-9b7e-09e75db31f06 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:24.040969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:24.046138 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7db7d-f505-4947-9649-321cf94602c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:24.047240 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7db7d-f505-4947-9649-321cf94602c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:24.047525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7db7d-f505-4947-9649-321cf94602c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:24.048011 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7db7d-f505-4947-9649-321cf94602c5 admin admin] PATH_INFO: `/v3/projects/2f8a894e68d04bf589a9b42b6a78e64e/users/734d0448ba5c4c0e86ab1feb6c871fb8/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:24.064954 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd7db7d-f505-4947-9649-321cf94602c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f8a894e68d04bf589a9b42b6a78e64e, user_id=734d0448ba5c4c0e86ab1feb6c871fb8, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:24.067721 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd7db7d-f505-4947-9649-321cf94602c5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:24.080809 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 814/1605] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:04:23 2022] PUT /identity/v3/projects/2f8a894e68d04bf589a9b42b6a78e64e/users/734d0448ba5c4c0e86ab1feb6c871fb8/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:24.081615 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7db7d-f505-4947-9649-321cf94602c5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:24.095086 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0596b18-8bcd-4eff-8059-e7cc4645d2e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:24.096612 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0596b18-8bcd-4eff-8059-e7cc4645d2e1 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:24.107771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 792/1606] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:04:24.108567 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0596b18-8bcd-4eff-8059-e7cc4645d2e1 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:24.121694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0596b18-8bcd-4eff-8059-e7cc4645d2e1 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:24.143859 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7db7d-f505-4947-9649-321cf94602c5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:24.148529 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9008e-edc3-410a-924e-200157ef0a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:24.149462 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9008e-edc3-410a-924e-200157ef0a1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:24.149758 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9008e-edc3-410a-924e-200157ef0a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:24.150027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9008e-edc3-410a-924e-200157ef0a1b admin admin] PATH_INFO: `/v3/projects/d6b449a0c00f40b7898872ceabe812cf/users/761b320197dc46a4a517cf7e56276c09/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:24.166664 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e9008e-edc3-410a-924e-200157ef0a1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6b449a0c00f40b7898872ceabe812cf, user_id=761b320197dc46a4a517cf7e56276c09, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:24.169333 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e9008e-edc3-410a-924e-200157ef0a1b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:24.184901 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 815/1607] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:04:24 2022] PUT /identity/v3/projects/d6b449a0c00f40b7898872ceabe812cf/users/761b320197dc46a4a517cf7e56276c09/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:24.186085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9008e-edc3-410a-924e-200157ef0a1b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:24.191901 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe50b32-111a-4f76-8734-ef35610f1089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:24.192812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe50b32-111a-4f76-8734-ef35610f1089 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:24.193099 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe50b32-111a-4f76-8734-ef35610f1089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:24.193389 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe50b32-111a-4f76-8734-ef35610f1089 admin admin] PATH_INFO: `/v3/projects/dc6bd5cfc57341a8bf462c206985466b/users/294e5f5819c741b38a721ab84d6b8613/roles/54195e660fc349978b39d5794765a095` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:24.211235 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe50b32-111a-4f76-8734-ef35610f1089 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc6bd5cfc57341a8bf462c206985466b, user_id=294e5f5819c741b38a721ab84d6b8613, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:24.214030 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe50b32-111a-4f76-8734-ef35610f1089 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:24.229083 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 793/1608] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:04:24 2022] PUT /identity/v3/projects/dc6bd5cfc57341a8bf462c206985466b/users/294e5f5819c741b38a721ab84d6b8613/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:24.239878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe50b32-111a-4f76-8734-ef35610f1089 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:24.262150 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9008e-edc3-410a-924e-200157ef0a1b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:24.307200 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe50b32-111a-4f76-8734-ef35610f1089 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:24.315979 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23390e93-cfbe-4b3a-82f5-c891906a5ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:24.317003 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23390e93-cfbe-4b3a-82f5-c891906a5ca5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:24.317508 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23390e93-cfbe-4b3a-82f5-c891906a5ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:24.317935 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23390e93-cfbe-4b3a-82f5-c891906a5ca5 admin admin] PATH_INFO: `/v3/projects/535c43e553a144869d08a61dd9c086a4/users/3f984e42c1d3464f94ea9693a9eeb55a/roles/54195e660fc349978b39d5794765a095` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:24.330538 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8b87f-6aee-4df7-a347-27de584215c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:24.331221 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8b87f-6aee-4df7-a347-27de584215c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:24.331507 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8b87f-6aee-4df7-a347-27de584215c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:24.332423 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8b87f-6aee-4df7-a347-27de584215c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:24.333188 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c8b87f-6aee-4df7-a347-27de584215c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:24.336625 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c8b87f-6aee-4df7-a347-27de584215c4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:24.338864 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23390e93-cfbe-4b3a-82f5-c891906a5ca5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=535c43e553a144869d08a61dd9c086a4, user_id=3f984e42c1d3464f94ea9693a9eeb55a, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:24.342211 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23390e93-cfbe-4b3a-82f5-c891906a5ca5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:24.348699 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 816/1609] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:24.349872 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8b87f-6aee-4df7-a347-27de584215c4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:24.358545 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 794/1610] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:04:24 2022] PUT /identity/v3/projects/535c43e553a144869d08a61dd9c086a4/users/3f984e42c1d3464f94ea9693a9eeb55a/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:24.360215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffb338-67c5-4250-a0a0-85c9f05db0ae None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:24.360582 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffb338-67c5-4250-a0a0-85c9f05db0ae None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:24.360863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffb338-67c5-4250-a0a0-85c9f05db0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:24.367995 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8b87f-6aee-4df7-a347-27de584215c4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:24.373093 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:24.391119 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-78ffb338-67c5-4250-a0a0-85c9f05db0ae None None] MFA Rules not processed for user `2dbe3a4500974e57b875b8c10a62d10e`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:24.392362 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-78ffb338-67c5-4250-a0a0-85c9f05db0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:24.402688 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:24.444922 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cfc641-53f4-4d2f-8b0b-734e6c655026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:24.446159 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cfc641-53f4-4d2f-8b0b-734e6c655026 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:24.446477 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cfc641-53f4-4d2f-8b0b-734e6c655026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:24.446756 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cfc641-53f4-4d2f-8b0b-734e6c655026 admin admin] PATH_INFO: `/v3/projects/e57e443306ff4510b03a45d02e27a033/users/35805c9448d848628d872a94462b1d6b/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:24.463540 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cfc641-53f4-4d2f-8b0b-734e6c655026 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e57e443306ff4510b03a45d02e27a033, user_id=35805c9448d848628d872a94462b1d6b, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:24.466326 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cfc641-53f4-4d2f-8b0b-734e6c655026 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:24.476713 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 795/1611] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:24 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:24.481851 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175e4f7-4e04-4871-9793-7be4829f9bf0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:24.481851 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175e4f7-4e04-4871-9793-7be4829f9bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:24.481851 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175e4f7-4e04-4871-9793-7be4829f9bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:24.495386 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 817/1612] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:04:24 2022] PUT /identity/v3/projects/e57e443306ff4510b03a45d02e27a033/users/35805c9448d848628d872a94462b1d6b/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:24.497984 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b62b4-5de7-408c-a579-aa114a117380 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:24.498387 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b62b4-5de7-408c-a579-aa114a117380 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:24.498798 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b62b4-5de7-408c-a579-aa114a117380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:24.523021 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a175e4f7-4e04-4871-9793-7be4829f9bf0 None None] MFA Rules not processed for user `734d0448ba5c4c0e86ab1feb6c871fb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:24.524437 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a175e4f7-4e04-4871-9793-7be4829f9bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:24.531289 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-629b62b4-5de7-408c-a579-aa114a117380 None None] MFA Rules not processed for user `761b320197dc46a4a517cf7e56276c09`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:24.532636 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-629b62b4-5de7-408c-a579-aa114a117380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:24.610285 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 796/1613] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:24.612024 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe85c16-1254-4209-80a6-45548fba4c96 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:24.612675 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe85c16-1254-4209-80a6-45548fba4c96 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:24.613160 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe85c16-1254-4209-80a6-45548fba4c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:24.619210 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 818/1614] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:24.619210 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b62b4-5de7-408c-a579-aa114a117380 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:24.647019 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-cfe85c16-1254-4209-80a6-45548fba4c96 None None] MFA Rules not processed for user `294e5f5819c741b38a721ab84d6b8613`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:24.648781 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cfe85c16-1254-4209-80a6-45548fba4c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:24.699615 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b62b4-5de7-408c-a579-aa114a117380 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:24.704964 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecda954-563a-44ec-ba5b-7d3dca76069b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:24.706040 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecda954-563a-44ec-ba5b-7d3dca76069b admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:24.706478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecda954-563a-44ec-ba5b-7d3dca76069b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:24.706739 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecda954-563a-44ec-ba5b-7d3dca76069b admin admin] PATH_INFO: `/v3/projects/c876b26300114f33b915345c7dc524e2/users/05afc537455c4a79be9fc8c4fdb1fc96/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:24.724418 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 797/1615] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:24.725588 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecda954-563a-44ec-ba5b-7d3dca76069b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c876b26300114f33b915345c7dc524e2, user_id=05afc537455c4a79be9fc8c4fdb1fc96, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:24.726122 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa627a-2aa9-4ed1-aa67-cc0e09304872 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:24.726485 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa627a-2aa9-4ed1-aa67-cc0e09304872 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:24.726837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa627a-2aa9-4ed1-aa67-cc0e09304872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:24.729269 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecda954-563a-44ec-ba5b-7d3dca76069b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:24.746441 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a1aa627a-2aa9-4ed1-aa67-cc0e09304872 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:24.747228 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 819/1616] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:04:24 2022] PUT /identity/v3/projects/c876b26300114f33b915345c7dc524e2/users/05afc537455c4a79be9fc8c4fdb1fc96/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:24.747811 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a1aa627a-2aa9-4ed1-aa67-cc0e09304872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:24.747995 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecda954-563a-44ec-ba5b-7d3dca76069b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:24.753724 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:24.767622 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:24.811408 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecda954-563a-44ec-ba5b-7d3dca76069b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:24.815723 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ede77f-dad1-4fad-b130-5f2c827cd45a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:24.816582 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ede77f-dad1-4fad-b130-5f2c827cd45a service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:24.816873 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ede77f-dad1-4fad-b130-5f2c827cd45a service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:24.817612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ede77f-dad1-4fad-b130-5f2c827cd45a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:24.819159 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 798/1617] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:24.820575 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899249fc-aa37-4ad6-8a9b-62d73841384d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:24.820691 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899249fc-aa37-4ad6-8a9b-62d73841384d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:24.820955 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899249fc-aa37-4ad6-8a9b-62d73841384d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:24.854585 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-899249fc-aa37-4ad6-8a9b-62d73841384d None None] MFA Rules not processed for user `35805c9448d848628d872a94462b1d6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:24.855627 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-899249fc-aa37-4ad6-8a9b-62d73841384d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:24.888025 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ede77f-dad1-4fad-b130-5f2c827cd45a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:24.889634 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ede77f-dad1-4fad-b130-5f2c827cd45a service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:24.900766 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 820/1618] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:24 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:04:24.901546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ede77f-dad1-4fad-b130-5f2c827cd45a service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:24.912640 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ede77f-dad1-4fad-b130-5f2c827cd45a service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:24.922414 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5edb4f-dc47-4ea4-a363-b67b04942132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:24.923127 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5edb4f-dc47-4ea4-a363-b67b04942132 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:24.923415 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5edb4f-dc47-4ea4-a363-b67b04942132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:24.923691 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5edb4f-dc47-4ea4-a363-b67b04942132 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:24.929564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5edb4f-dc47-4ea4-a363-b67b04942132 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:24.931926 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5edb4f-dc47-4ea4-a363-b67b04942132 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:24.957071 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 799/1619] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:24 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:24.958561 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248cdb5-7d4c-4566-9adc-2cb2b2d751f4 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:24.958855 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248cdb5-7d4c-4566-9adc-2cb2b2d751f4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:24.959138 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248cdb5-7d4c-4566-9adc-2cb2b2d751f4 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:24.960694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 800/1620] 10.222.0.52 () {54 vars in 962 bytes} [Thu Jul 21 16:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 16:04:24.969986 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248cdb5-7d4c-4566-9adc-2cb2b2d751f4 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:24.996925 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 821/1621] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:04:24 2022] POST /identity/v3/projects => generated 379 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:24.998688 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d7cc9c-6733-4d81-9996-a75775467e7d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:24.998971 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d7cc9c-6733-4d81-9996-a75775467e7d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:24.999266 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d7cc9c-6733-4d81-9996-a75775467e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:25.002021 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248cdb5-7d4c-4566-9adc-2cb2b2d751f4 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:25.036533 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-69d7cc9c-6733-4d81-9996-a75775467e7d None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:25.037762 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-69d7cc9c-6733-4d81-9996-a75775467e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:25.094662 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2eaab0-9549-4ad3-9b9a-0860eae5ab20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:25.095476 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2eaab0-9549-4ad3-9b9a-0860eae5ab20 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:25.095830 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2eaab0-9549-4ad3-9b9a-0860eae5ab20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:25.096179 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2eaab0-9549-4ad3-9b9a-0860eae5ab20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:25.097043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2eaab0-9549-4ad3-9b9a-0860eae5ab20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:25.099966 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2eaab0-9549-4ad3-9b9a-0860eae5ab20 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:25.124563 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 822/1622] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:25.127019 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d7cc9c-6733-4d81-9996-a75775467e7d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:25.135577 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 801/1623] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:04:24 2022] POST /identity/v3/projects => generated 373 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:25.136756 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e0308-f9c5-425b-bb70-99202e4dbc1b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:25.137316 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e0308-f9c5-425b-bb70-99202e4dbc1b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:25.137611 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e0308-f9c5-425b-bb70-99202e4dbc1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:25.173137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:25.181781 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3f0e0308-f9c5-425b-bb70-99202e4dbc1b None None] MFA Rules not processed for user `092553a86f8848dba8c7271f5c0b2c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:25.183990 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3f0e0308-f9c5-425b-bb70-99202e4dbc1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:25.207191 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d7cc9c-6733-4d81-9996-a75775467e7d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:25.208432 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:25.271725 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 802/1624] 10.222.0.52 () {58 vars in 1059 bytes} [Thu Jul 21 16:04:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:25.272983 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e0308-f9c5-425b-bb70-99202e4dbc1b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:25.292435 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36703170-9919-4e10-8ee8-45f6e1219ae1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:25.293091 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36703170-9919-4e10-8ee8-45f6e1219ae1 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:25.293372 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36703170-9919-4e10-8ee8-45f6e1219ae1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:25.294235 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36703170-9919-4e10-8ee8-45f6e1219ae1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:25.345427 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e0308-f9c5-425b-bb70-99202e4dbc1b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:25.350241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e14c5-de86-48ba-966e-89d5a82a7358 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:25.351276 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e14c5-de86-48ba-966e-89d5a82a7358 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:25.351781 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e14c5-de86-48ba-966e-89d5a82a7358 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:25.352290 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e14c5-de86-48ba-966e-89d5a82a7358 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:25.363936 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36703170-9919-4e10-8ee8-45f6e1219ae1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:25.365694 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36703170-9919-4e10-8ee8-45f6e1219ae1 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:25.376658 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 823/1625] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:04:25.377498 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36703170-9919-4e10-8ee8-45f6e1219ae1 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:25.388182 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36703170-9919-4e10-8ee8-45f6e1219ae1 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:25.392902 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e50452-0ea4-4f4e-8f43-113517e5ec41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:25.393718 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e50452-0ea4-4f4e-8f43-113517e5ec41 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:25.394003 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e50452-0ea4-4f4e-8f43-113517e5ec41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:25.394277 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e50452-0ea4-4f4e-8f43-113517e5ec41 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:25.395116 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e50452-0ea4-4f4e-8f43-113517e5ec41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:25.397278 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e50452-0ea4-4f4e-8f43-113517e5ec41 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:25.429692 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 824/1626] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:04:25 2022] POST /identity/v3/users => generated 296 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:25.432282 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e50452-0ea4-4f4e-8f43-113517e5ec41 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:25.445509 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e50452-0ea4-4f4e-8f43-113517e5ec41 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:25.449998 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff3d4fb-3a18-43e3-9e8b-1e8ee6b315ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:25.450642 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3d4fb-3a18-43e3-9e8b-1e8ee6b315ae service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:25.454657 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3d4fb-3a18-43e3-9e8b-1e8ee6b315ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:25.454657 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3d4fb-3a18-43e3-9e8b-1e8ee6b315ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:25.464578 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7e14c5-de86-48ba-966e-89d5a82a7358 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:25.467622 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7e14c5-de86-48ba-966e-89d5a82a7358 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:25.504757 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 803/1627] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:25 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:04:25.507143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e14c5-de86-48ba-966e-89d5a82a7358 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:25.512664 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5b7e14c5-de86-48ba-966e-89d5a82a7358 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:25.546836 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e14c5-de86-48ba-966e-89d5a82a7358 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:25.558306 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef492ef-9dd2-4dcf-9415-4f571ac6ad4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:25.558877 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff3d4fb-3a18-43e3-9e8b-1e8ee6b315ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:25.560389 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef492ef-9dd2-4dcf-9415-4f571ac6ad4a service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:25.560911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff3d4fb-3a18-43e3-9e8b-1e8ee6b315ae service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:25.563001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef492ef-9dd2-4dcf-9415-4f571ac6ad4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:25.563627 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef492ef-9dd2-4dcf-9415-4f571ac6ad4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:25.572590 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 825/1628] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:04:25.573495 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff3d4fb-3a18-43e3-9e8b-1e8ee6b315ae service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:25.588739 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff3d4fb-3a18-43e3-9e8b-1e8ee6b315ae service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:25.593903 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86705505-edeb-4b7e-95ad-97becff8580c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:25.594579 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86705505-edeb-4b7e-95ad-97becff8580c admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:25.595047 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86705505-edeb-4b7e-95ad-97becff8580c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:25.595327 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86705505-edeb-4b7e-95ad-97becff8580c admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:25.595575 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef492ef-9dd2-4dcf-9415-4f571ac6ad4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:25.596020 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86705505-edeb-4b7e-95ad-97becff8580c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:25.598491 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86705505-edeb-4b7e-95ad-97becff8580c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:25.600862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef492ef-9dd2-4dcf-9415-4f571ac6ad4a service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:25.621910 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:25.635751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 826/1629] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:04:25 2022] POST /identity/v3/users => generated 293 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:25.637111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86705505-edeb-4b7e-95ad-97becff8580c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:25.657050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 804/1630] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:04:25.671852 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef492ef-9dd2-4dcf-9415-4f571ac6ad4a service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:25.672077 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86705505-edeb-4b7e-95ad-97becff8580c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:25.692650 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a063c4-7132-4cde-915c-9891469d2583 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:25.694211 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a063c4-7132-4cde-915c-9891469d2583 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:25.695396 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a063c4-7132-4cde-915c-9891469d2583 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:25.697118 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a063c4-7132-4cde-915c-9891469d2583 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:25.720608 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef492ef-9dd2-4dcf-9415-4f571ac6ad4a service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:25.729345 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21fb57e-ea5f-4f73-9b6e-e48059d5e243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:25.730779 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fb57e-ea5f-4f73-9b6e-e48059d5e243 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:25.732134 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fb57e-ea5f-4f73-9b6e-e48059d5e243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:25.732134 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fb57e-ea5f-4f73-9b6e-e48059d5e243 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:25.735272 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21fb57e-ea5f-4f73-9b6e-e48059d5e243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:25.739165 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21fb57e-ea5f-4f73-9b6e-e48059d5e243 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:25.758409 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 805/1631] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:25.760218 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21fb57e-ea5f-4f73-9b6e-e48059d5e243 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:25.793801 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21fb57e-ea5f-4f73-9b6e-e48059d5e243 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:25.801685 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40060d88-1063-4843-8eb2-e71764f929a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:25.802574 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40060d88-1063-4843-8eb2-e71764f929a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:25.802877 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40060d88-1063-4843-8eb2-e71764f929a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:25.803146 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40060d88-1063-4843-8eb2-e71764f929a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:25.804259 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40060d88-1063-4843-8eb2-e71764f929a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:25.807064 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40060d88-1063-4843-8eb2-e71764f929a7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:25.832863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 806/1632] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:25.832863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40060d88-1063-4843-8eb2-e71764f929a7 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:25.864699 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40060d88-1063-4843-8eb2-e71764f929a7 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:25.880342 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0020a82-f260-4f1e-8c37-0300ece1b34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:25.886045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a063c4-7132-4cde-915c-9891469d2583 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:25.888639 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a063c4-7132-4cde-915c-9891469d2583 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:25.888771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0020a82-f260-4f1e-8c37-0300ece1b34c admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:25.889112 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0020a82-f260-4f1e-8c37-0300ece1b34c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:25.889403 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0020a82-f260-4f1e-8c37-0300ece1b34c admin admin] PATH_INFO: `/v3/projects/5bb03f8c149f4135adbaabc2102e6d59/users/9bdc10d2d386413b841833f72fa557da/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:25.925358 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 827/1633] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:04:25.926749 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a063c4-7132-4cde-915c-9891469d2583 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:25.932183 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0020a82-f260-4f1e-8c37-0300ece1b34c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bb03f8c149f4135adbaabc2102e6d59, user_id=9bdc10d2d386413b841833f72fa557da, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:25.934898 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0020a82-f260-4f1e-8c37-0300ece1b34c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:25.961060 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a063c4-7132-4cde-915c-9891469d2583 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:25.984830 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 807/1634] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:04:25 2022] PUT /identity/v3/projects/5bb03f8c149f4135adbaabc2102e6d59/users/9bdc10d2d386413b841833f72fa557da/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:25.988807 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0020a82-f260-4f1e-8c37-0300ece1b34c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:26.045440 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:26.163868 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0020a82-f260-4f1e-8c37-0300ece1b34c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:26.173456 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ae041-671e-4571-a467-0f889aadde29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:26.174643 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ae041-671e-4571-a467-0f889aadde29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:26.176419 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ae041-671e-4571-a467-0f889aadde29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:26.177891 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ae041-671e-4571-a467-0f889aadde29 admin admin] PATH_INFO: `/v3/projects/05269b3d4b324a4bb77f16da40c790b4/users/e2daefa8f19c4284bee27b8e961ad7d1/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:26.194418 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8069102-0f3a-43da-8099-3ac533e4ff06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:26.195142 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8069102-0f3a-43da-8099-3ac533e4ff06 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:26.195436 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8069102-0f3a-43da-8099-3ac533e4ff06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:26.197209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8069102-0f3a-43da-8099-3ac533e4ff06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:26.197209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8069102-0f3a-43da-8099-3ac533e4ff06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:26.198694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8069102-0f3a-43da-8099-3ac533e4ff06 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:26.214690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ae041-671e-4571-a467-0f889aadde29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05269b3d4b324a4bb77f16da40c790b4, user_id=e2daefa8f19c4284bee27b8e961ad7d1, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:26.221985 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 808/1635] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:26.224081 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8069102-0f3a-43da-8099-3ac533e4ff06 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:26.227897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ae041-671e-4571-a467-0f889aadde29 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:26.258360 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8069102-0f3a-43da-8099-3ac533e4ff06 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:26.272918 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 828/1636] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:04:25 2022] PUT /identity/v3/projects/05269b3d4b324a4bb77f16da40c790b4/users/e2daefa8f19c4284bee27b8e961ad7d1/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:26.275658 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a48ed-d5ce-4306-8254-5a945d6862c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:26.276297 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ae041-671e-4571-a467-0f889aadde29 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:26.276450 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a48ed-d5ce-4306-8254-5a945d6862c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:26.276744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a48ed-d5ce-4306-8254-5a945d6862c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:26.277030 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a48ed-d5ce-4306-8254-5a945d6862c9 admin admin] PATH_INFO: `/v3/projects/5bb03f8c149f4135adbaabc2102e6d59/users/9bdc10d2d386413b841833f72fa557da/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:26.279185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a48ed-d5ce-4306-8254-5a945d6862c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bb03f8c149f4135adbaabc2102e6d59, user_id=9bdc10d2d386413b841833f72fa557da, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:26.282054 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a48ed-d5ce-4306-8254-5a945d6862c9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:26.310944 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 809/1637] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:04:26 2022] PUT /identity/v3/projects/5bb03f8c149f4135adbaabc2102e6d59/users/9bdc10d2d386413b841833f72fa557da/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:26.321823 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833b249-eb47-42a4-afaa-bdcd772181c8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:26.322137 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833b249-eb47-42a4-afaa-bdcd772181c8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:26.322429 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833b249-eb47-42a4-afaa-bdcd772181c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:26.385068 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4833b249-eb47-42a4-afaa-bdcd772181c8 None None] MFA Rules not processed for user `9bdc10d2d386413b841833f72fa557da`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:26.386544 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4833b249-eb47-42a4-afaa-bdcd772181c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:26.531708 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ae041-671e-4571-a467-0f889aadde29 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:26.574747 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 810/1638] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:26 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:04:26.576824 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc826e6a-8d87-4ee7-a863-750e4c08c60a None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:26.577128 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc826e6a-8d87-4ee7-a863-750e4c08c60a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:26.577403 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc826e6a-8d87-4ee7-a863-750e4c08c60a None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:26.579327 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 811/1639] 10.222.0.52 () {54 vars in 912 bytes} [Thu Jul 21 16:04:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 16:04:26.580224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc826e6a-8d87-4ee7-a863-750e4c08c60a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:26.594296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:26.670263 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2abfd-f416-46d2-842e-e350fd54b3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:26.672326 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2abfd-f416-46d2-842e-e350fd54b3c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:26.673294 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2abfd-f416-46d2-842e-e350fd54b3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:26.673691 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2abfd-f416-46d2-842e-e350fd54b3c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:26.676968 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce2abfd-f416-46d2-842e-e350fd54b3c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:26.679862 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce2abfd-f416-46d2-842e-e350fd54b3c7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:26.705940 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc826e6a-8d87-4ee7-a863-750e4c08c60a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:26.706180 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 829/1640] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:26.707198 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db818a-b3b1-4db9-a564-4b43e6ef8b37 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:26.712097 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db818a-b3b1-4db9-a564-4b43e6ef8b37 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:26.712499 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db818a-b3b1-4db9-a564-4b43e6ef8b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:26.714481 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4f471-4ced-4103-9b19-793bc5b55f13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:26.715365 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4f471-4ced-4103-9b19-793bc5b55f13 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:26.716199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4f471-4ced-4103-9b19-793bc5b55f13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:26.716341 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4f471-4ced-4103-9b19-793bc5b55f13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:26.731301 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd4f471-4ced-4103-9b19-793bc5b55f13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:26.733678 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd4f471-4ced-4103-9b19-793bc5b55f13 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:26.741723 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-12db818a-b3b1-4db9-a564-4b43e6ef8b37 None None] MFA Rules not processed for user `092553a86f8848dba8c7271f5c0b2c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:26.743997 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-12db818a-b3b1-4db9-a564-4b43e6ef8b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:26.751406 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 812/1641] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:26 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 16:04:26.752973 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4f471-4ced-4103-9b19-793bc5b55f13 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:26.766263 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4f471-4ced-4103-9b19-793bc5b55f13 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:26.772649 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee32821d-aa15-45bd-96ba-9616aa31c0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:26.773946 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32821d-aa15-45bd-96ba-9616aa31c0c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:26.774240 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32821d-aa15-45bd-96ba-9616aa31c0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:26.774556 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32821d-aa15-45bd-96ba-9616aa31c0c0 admin admin] PATH_INFO: `/v3/projects/05269b3d4b324a4bb77f16da40c790b4/users/e2daefa8f19c4284bee27b8e961ad7d1/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:26.777068 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee32821d-aa15-45bd-96ba-9616aa31c0c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05269b3d4b324a4bb77f16da40c790b4, user_id=e2daefa8f19c4284bee27b8e961ad7d1, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:26.780504 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee32821d-aa15-45bd-96ba-9616aa31c0c0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:26.803524 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 813/1642] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:04:26 2022] PUT /identity/v3/projects/05269b3d4b324a4bb77f16da40c790b4/users/e2daefa8f19c4284bee27b8e961ad7d1/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:26.808155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13c3d2-0e53-4da8-bdf4-e27801e2226b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:26.808626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13c3d2-0e53-4da8-bdf4-e27801e2226b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:26.808890 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13c3d2-0e53-4da8-bdf4-e27801e2226b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:26.834950 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 830/1643] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 21 16:04:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:26.872040 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:26.882622 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db818a-b3b1-4db9-a564-4b43e6ef8b37 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:26.898700 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db818a-b3b1-4db9-a564-4b43e6ef8b37 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:26.905095 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3b13c3d2-0e53-4da8-bdf4-e27801e2226b None None] MFA Rules not processed for user `e2daefa8f19c4284bee27b8e961ad7d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:26.906368 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3b13c3d2-0e53-4da8-bdf4-e27801e2226b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:27.023382 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd80de5-fd83-4e93-bef1-b3feb3c5c132 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:27.024430 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd80de5-fd83-4e93-bef1-b3feb3c5c132 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:27.024924 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd80de5-fd83-4e93-bef1-b3feb3c5c132 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:27.025376 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd80de5-fd83-4e93-bef1-b3feb3c5c132 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:27.026277 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 814/1644] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:26 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:27.164515 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b13c3d2-0e53-4da8-bdf4-e27801e2226b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:27.169085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd80de5-fd83-4e93-bef1-b3feb3c5c132 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:27.171252 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd80de5-fd83-4e93-bef1-b3feb3c5c132 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:27.180320 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b13c3d2-0e53-4da8-bdf4-e27801e2226b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:27.186881 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:27.190743 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481bcd38-1caf-498a-b288-57e74d6cdab2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:27.194427 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bcd38-1caf-498a-b288-57e74d6cdab2 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:27.195114 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bcd38-1caf-498a-b288-57e74d6cdab2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:27.195382 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bcd38-1caf-498a-b288-57e74d6cdab2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:27.221710 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 831/1645] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:04:27.329851 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481bcd38-1caf-498a-b288-57e74d6cdab2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:27.331332 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481bcd38-1caf-498a-b288-57e74d6cdab2 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:27.345414 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 815/1646] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 16:04:27.555167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd80de5-fd83-4e93-bef1-b3feb3c5c132 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:27.578169 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd80de5-fd83-4e93-bef1-b3feb3c5c132 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:27.585525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26beed5b-e324-4bd1-b3da-66cfe68850ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:27.590783 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26beed5b-e324-4bd1-b3da-66cfe68850ec service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:27.595174 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26beed5b-e324-4bd1-b3da-66cfe68850ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:27.595174 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26beed5b-e324-4bd1-b3da-66cfe68850ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:27.609440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26beed5b-e324-4bd1-b3da-66cfe68850ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:27.609910 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26beed5b-e324-4bd1-b3da-66cfe68850ec service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:27.625535 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 832/1647] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:27 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:04:28.649995 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f65491-83ff-4a00-a915-650a599950ec None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:28.650638 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f65491-83ff-4a00-a915-650a599950ec None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:28.650783 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f65491-83ff-4a00-a915-650a599950ec None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:28.652405 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 816/1648] 10.222.0.52 () {54 vars in 962 bytes} [Thu Jul 21 16:04:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 16:04:28.668866 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f4e96-c74c-4ef7-a1e0-050313bceb94 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:28.669357 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f4e96-c74c-4ef7-a1e0-050313bceb94 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:28.669454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f4e96-c74c-4ef7-a1e0-050313bceb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:28.718894 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7f2f4e96-c74c-4ef7-a1e0-050313bceb94 None None] MFA Rules not processed for user `092553a86f8848dba8c7271f5c0b2c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:28.722294 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7f2f4e96-c74c-4ef7-a1e0-050313bceb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:28.750263 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 833/1649] 10.222.0.52 () {58 vars in 1059 bytes} [Thu Jul 21 16:04:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:28.754713 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0235a-e153-4eef-8003-c1783109824e None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:28.755246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0235a-e153-4eef-8003-c1783109824e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:28.755799 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0235a-e153-4eef-8003-c1783109824e None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:28.757435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 817/1650] 10.222.0.52 () {54 vars in 912 bytes} [Thu Jul 21 16:04:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 16:04:28.764168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc915f4a-674d-4329-830d-cd1b8b49f8d2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:28.764668 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc915f4a-674d-4329-830d-cd1b8b49f8d2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:28.765116 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc915f4a-674d-4329-830d-cd1b8b49f8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:28.805271 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-dc915f4a-674d-4329-830d-cd1b8b49f8d2 None None] MFA Rules not processed for user `092553a86f8848dba8c7271f5c0b2c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:28.811117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-dc915f4a-674d-4329-830d-cd1b8b49f8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:28.836537 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 834/1651] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 21 16:04:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:29.075357 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0235a-e153-4eef-8003-c1783109824e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:29.099624 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0235a-e153-4eef-8003-c1783109824e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:29.106026 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a80e22-6649-4659-8147-9af6dc9af39a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:29.107132 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a80e22-6649-4659-8147-9af6dc9af39a service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:29.107653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a80e22-6649-4659-8147-9af6dc9af39a service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:29.108255 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a80e22-6649-4659-8147-9af6dc9af39a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:29.135297 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a80e22-6649-4659-8147-9af6dc9af39a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:29.137226 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a80e22-6649-4659-8147-9af6dc9af39a service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:29.155439 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 818/1652] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:04:30.632297 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc915f4a-674d-4329-830d-cd1b8b49f8d2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:30.781479 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc915f4a-674d-4329-830d-cd1b8b49f8d2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:30.787847 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d804744-e6d0-4ab9-88ee-af6c508bd2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:30.789780 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d804744-e6d0-4ab9-88ee-af6c508bd2fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:30.790253 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d804744-e6d0-4ab9-88ee-af6c508bd2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:30.790792 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d804744-e6d0-4ab9-88ee-af6c508bd2fb admin admin] PATH_INFO: `/v3/users/294e5f5819c741b38a721ab84d6b8613` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:30.792972 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d804744-e6d0-4ab9-88ee-af6c508bd2fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=294e5f5819c741b38a721ab84d6b8613)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:30.795002 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-6d804744-e6d0-4ab9-88ee-af6c508bd2fb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 16:04:30.795002 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: warnings.warn(msg) Jul 21 16:04:30.795002 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: Jul 21 16:04:30.796244 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d804744-e6d0-4ab9-88ee-af6c508bd2fb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:30.973518 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6d804744-e6d0-4ab9-88ee-af6c508bd2fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '294e5f5819c741b38a721ab84d6b8613'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:30.974364 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6d804744-e6d0-4ab9-88ee-af6c508bd2fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '294e5f5819c741b38a721ab84d6b8613'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:30.974987 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6d804744-e6d0-4ab9-88ee-af6c508bd2fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '294e5f5819c741b38a721ab84d6b8613'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:30.991420 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6d804744-e6d0-4ab9-88ee-af6c508bd2fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '294e5f5819c741b38a721ab84d6b8613'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:31.001469 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6d804744-e6d0-4ab9-88ee-af6c508bd2fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '294e5f5819c741b38a721ab84d6b8613'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:31.077933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 835/1653] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:04:30 2022] DELETE /identity/v3/users/294e5f5819c741b38a721ab84d6b8613 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:31.083303 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2bc6b7-b625-4bb9-8ada-c74bd0fde1d3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:31.083972 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2bc6b7-b625-4bb9-8ada-c74bd0fde1d3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:31.083972 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2bc6b7-b625-4bb9-8ada-c74bd0fde1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:31.142211 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-db2bc6b7-b625-4bb9-8ada-c74bd0fde1d3 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:31.151272 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-db2bc6b7-b625-4bb9-8ada-c74bd0fde1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:31.331494 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 819/1654] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:31.349527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d804744-e6d0-4ab9-88ee-af6c508bd2fb admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:31.350932 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6d804744-e6d0-4ab9-88ee-af6c508bd2fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:31.476240 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d804744-e6d0-4ab9-88ee-af6c508bd2fb admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:31.480637 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6209d6c2-4056-400e-80a0-8f7beecb602d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:31.481315 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209d6c2-4056-400e-80a0-8f7beecb602d service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:31.481619 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209d6c2-4056-400e-80a0-8f7beecb602d service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:31.481895 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209d6c2-4056-400e-80a0-8f7beecb602d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:31.498384 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:31.520652 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6209d6c2-4056-400e-80a0-8f7beecb602d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:31.522278 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6209d6c2-4056-400e-80a0-8f7beecb602d service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:31.549596 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 836/1655] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:04:32.859564 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2bc6b7-b625-4bb9-8ada-c74bd0fde1d3 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:32.863859 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-db2bc6b7-b625-4bb9-8ada-c74bd0fde1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:32.880436 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2bc6b7-b625-4bb9-8ada-c74bd0fde1d3 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:32.903408 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942e1e82-ad80-407e-b710-4eecbee81e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:32.904198 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e1e82-ad80-407e-b710-4eecbee81e0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:32.904607 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e1e82-ad80-407e-b710-4eecbee81e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:32.904774 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e1e82-ad80-407e-b710-4eecbee81e0e admin admin] PATH_INFO: `/v3/projects/dc6bd5cfc57341a8bf462c206985466b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:32.906296 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942e1e82-ad80-407e-b710-4eecbee81e0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc6bd5cfc57341a8bf462c206985466b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:32.907370 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: WARNING py.warnings [None req-942e1e82-ad80-407e-b710-4eecbee81e0e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 16:04:32.907370 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: warnings.warn(msg) Jul 21 16:04:32.907370 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: Jul 21 16:04:32.908597 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942e1e82-ad80-407e-b710-4eecbee81e0e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:32.968619 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:33.049056 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-942e1e82-ad80-407e-b710-4eecbee81e0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc6bd5cfc57341a8bf462c206985466b', 'request_id': 'req-942e1e82-ad80-407e-b710-4eecbee81e0e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:33.086356 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-942e1e82-ad80-407e-b710-4eecbee81e0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc6bd5cfc57341a8bf462c206985466b', 'request_id': 'req-942e1e82-ad80-407e-b710-4eecbee81e0e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:33.101857 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-942e1e82-ad80-407e-b710-4eecbee81e0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc6bd5cfc57341a8bf462c206985466b', 'request_id': 'req-942e1e82-ad80-407e-b710-4eecbee81e0e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:33.122581 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-942e1e82-ad80-407e-b710-4eecbee81e0e admin admin] The token cache is being invalidate because project dc6bd5cfc57341a8bf462c206985466b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:04:33.123297 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-942e1e82-ad80-407e-b710-4eecbee81e0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:33.124152 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-942e1e82-ad80-407e-b710-4eecbee81e0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:33.126575 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 820/1656] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:04:32 2022] DELETE /identity/v3/projects/dc6bd5cfc57341a8bf462c206985466b => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:33.130474 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6209d6c2-4056-400e-80a0-8f7beecb602d service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:33.143689 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6209d6c2-4056-400e-80a0-8f7beecb602d service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:33.275464 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0fa14c-2a04-4aae-b60f-06b3c7842c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:33.276459 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fa14c-2a04-4aae-b60f-06b3c7842c93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:33.277390 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fa14c-2a04-4aae-b60f-06b3c7842c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:33.277809 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fa14c-2a04-4aae-b60f-06b3c7842c93 admin admin] PATH_INFO: `/v3/users/e2daefa8f19c4284bee27b8e961ad7d1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:33.279704 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0fa14c-2a04-4aae-b60f-06b3c7842c93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2daefa8f19c4284bee27b8e961ad7d1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:33.281794 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0fa14c-2a04-4aae-b60f-06b3c7842c93 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:33.403143 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2a0fa14c-2a04-4aae-b60f-06b3c7842c93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2daefa8f19c4284bee27b8e961ad7d1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:33.404160 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2a0fa14c-2a04-4aae-b60f-06b3c7842c93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2daefa8f19c4284bee27b8e961ad7d1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:33.404987 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2a0fa14c-2a04-4aae-b60f-06b3c7842c93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2daefa8f19c4284bee27b8e961ad7d1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:33.419892 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2a0fa14c-2a04-4aae-b60f-06b3c7842c93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2daefa8f19c4284bee27b8e961ad7d1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:33.436584 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2a0fa14c-2a04-4aae-b60f-06b3c7842c93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2daefa8f19c4284bee27b8e961ad7d1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:33.470022 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 837/1657] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:04:33 2022] DELETE /identity/v3/users/e2daefa8f19c4284bee27b8e961ad7d1 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:34.217982 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d7e2c-a19f-4848-8832-0c01ec75b942 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:34.218433 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d7e2c-a19f-4848-8832-0c01ec75b942 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:34.218543 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d7e2c-a19f-4848-8832-0c01ec75b942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:34.296494 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-973d7e2c-a19f-4848-8832-0c01ec75b942 None None] MFA Rules not processed for user `a50cbf92286c420f9bee6e05c33dc440`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:34.297977 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-973d7e2c-a19f-4848-8832-0c01ec75b942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:34.453323 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 821/1658] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:34.469792 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0fa14c-2a04-4aae-b60f-06b3c7842c93 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:34.472217 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2a0fa14c-2a04-4aae-b60f-06b3c7842c93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:34.557092 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d7e2c-a19f-4848-8832-0c01ec75b942 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:34.558571 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-973d7e2c-a19f-4848-8832-0c01ec75b942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:34.569627 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0fa14c-2a04-4aae-b60f-06b3c7842c93 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:34.577356 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f8bbc5-bc89-4374-a125-10111e6d3b9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:34.578458 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8bbc5-bc89-4374-a125-10111e6d3b9d service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:34.578880 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8bbc5-bc89-4374-a125-10111e6d3b9d service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:34.579298 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8bbc5-bc89-4374-a125-10111e6d3b9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:34.590975 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f8bbc5-bc89-4374-a125-10111e6d3b9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:34.593010 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f8bbc5-bc89-4374-a125-10111e6d3b9d service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:34.598285 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:34.619588 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 838/1659] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:04:34.713059 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d7e2c-a19f-4848-8832-0c01ec75b942 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:34.721171 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73130fc5-2f61-450e-9b1e-765654bfc62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:34.722290 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73130fc5-2f61-450e-9b1e-765654bfc62a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:34.725972 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73130fc5-2f61-450e-9b1e-765654bfc62a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:34.726426 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73130fc5-2f61-450e-9b1e-765654bfc62a admin admin] PATH_INFO: `/v3/projects/05269b3d4b324a4bb77f16da40c790b4` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:34.728882 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73130fc5-2f61-450e-9b1e-765654bfc62a admin admin] RBAC: Authorizing `identity:delete_project(project_id=05269b3d4b324a4bb77f16da40c790b4)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:34.731014 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73130fc5-2f61-450e-9b1e-765654bfc62a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:34.804524 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:34.831605 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-73130fc5-2f61-450e-9b1e-765654bfc62a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05269b3d4b324a4bb77f16da40c790b4', 'request_id': 'req-73130fc5-2f61-450e-9b1e-765654bfc62a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:34.846985 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-73130fc5-2f61-450e-9b1e-765654bfc62a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05269b3d4b324a4bb77f16da40c790b4', 'request_id': 'req-73130fc5-2f61-450e-9b1e-765654bfc62a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:34.868354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-73130fc5-2f61-450e-9b1e-765654bfc62a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05269b3d4b324a4bb77f16da40c790b4', 'request_id': 'req-73130fc5-2f61-450e-9b1e-765654bfc62a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:34.885124 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-73130fc5-2f61-450e-9b1e-765654bfc62a admin admin] The token cache is being invalidate because project 05269b3d4b324a4bb77f16da40c790b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:04:34.885463 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-73130fc5-2f61-450e-9b1e-765654bfc62a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:34.886148 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-73130fc5-2f61-450e-9b1e-765654bfc62a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:34.889833 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 822/1660] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:04:34 2022] DELETE /identity/v3/projects/05269b3d4b324a4bb77f16da40c790b4 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:04:34.894681 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2157089-7a9e-41f3-ac5d-c4d234dc7f29 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:34.894978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2157089-7a9e-41f3-ac5d-c4d234dc7f29 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:34.895265 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2157089-7a9e-41f3-ac5d-c4d234dc7f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:34.938122 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d2157089-7a9e-41f3-ac5d-c4d234dc7f29 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:34.942249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d2157089-7a9e-41f3-ac5d-c4d234dc7f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:35.126350 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 839/1661] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:35.135649 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5f559-c12c-4d21-93e3-3828d9450b29 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:35.136301 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5f559-c12c-4d21-93e3-3828d9450b29 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:35.136658 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5f559-c12c-4d21-93e3-3828d9450b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:35.191093 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6cd5f559-c12c-4d21-93e3-3828d9450b29 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:35.198757 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6cd5f559-c12c-4d21-93e3-3828d9450b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:35.224967 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 823/1662] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:35.228958 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2157089-7a9e-41f3-ac5d-c4d234dc7f29 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:35.242643 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2157089-7a9e-41f3-ac5d-c4d234dc7f29 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:35.260207 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a18b50-e008-4054-947d-ecfde34e4855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:35.261378 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a18b50-e008-4054-947d-ecfde34e4855 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:35.261880 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a18b50-e008-4054-947d-ecfde34e4855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:35.262333 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a18b50-e008-4054-947d-ecfde34e4855 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:35.263281 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a18b50-e008-4054-947d-ecfde34e4855 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:04:35.263767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a18b50-e008-4054-947d-ecfde34e4855 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:35.269233 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a18b50-e008-4054-947d-ecfde34e4855 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:35.299980 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 840/1663] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:04:35.306166 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5f559-c12c-4d21-93e3-3828d9450b29 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:35.330345 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5f559-c12c-4d21-93e3-3828d9450b29 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:35.354190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f61900d-524d-4a58-98ac-d6eafef31bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:35.357063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61900d-524d-4a58-98ac-d6eafef31bff admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:35.357568 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61900d-524d-4a58-98ac-d6eafef31bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:35.358078 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61900d-524d-4a58-98ac-d6eafef31bff admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:35.360678 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f61900d-524d-4a58-98ac-d6eafef31bff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:35.365678 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f61900d-524d-4a58-98ac-d6eafef31bff admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:35.434292 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 824/1664] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:04:35 2022] POST /identity/v3/projects => generated 363 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:35.438445 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a18b50-e008-4054-947d-ecfde34e4855 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:35.455202 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a18b50-e008-4054-947d-ecfde34e4855 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:35.594251 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a711c92-2fdf-48e0-91d4-918929da0414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:35.595138 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a711c92-2fdf-48e0-91d4-918929da0414 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:35.595493 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a711c92-2fdf-48e0-91d4-918929da0414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:35.595974 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a711c92-2fdf-48e0-91d4-918929da0414 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:35.596999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a711c92-2fdf-48e0-91d4-918929da0414 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:35.599693 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a711c92-2fdf-48e0-91d4-918929da0414 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:35.633979 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 841/1665] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:04:35 2022] POST /identity/v3/users => generated 282 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:35.636079 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f61900d-524d-4a58-98ac-d6eafef31bff admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:35.657345 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f61900d-524d-4a58-98ac-d6eafef31bff admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:35.673746 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc65d4-12a2-442f-9a8e-15f338c6dd1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:35.674786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc65d4-12a2-442f-9a8e-15f338c6dd1b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:35.675297 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc65d4-12a2-442f-9a8e-15f338c6dd1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:35.675735 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc65d4-12a2-442f-9a8e-15f338c6dd1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:35.677293 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdc65d4-12a2-442f-9a8e-15f338c6dd1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:35.683542 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdc65d4-12a2-442f-9a8e-15f338c6dd1b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:35.716527 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 825/1666] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:35.717771 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a711c92-2fdf-48e0-91d4-918929da0414 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:35.735821 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a711c92-2fdf-48e0-91d4-918929da0414 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:35.754310 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730ced7-4441-4b32-abe5-6eb85c7eca8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:35.760197 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730ced7-4441-4b32-abe5-6eb85c7eca8a admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:35.760930 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730ced7-4441-4b32-abe5-6eb85c7eca8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:35.761886 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730ced7-4441-4b32-abe5-6eb85c7eca8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:35.765198 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8730ced7-4441-4b32-abe5-6eb85c7eca8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:35.768481 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8730ced7-4441-4b32-abe5-6eb85c7eca8a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:35.806184 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 842/1667] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:35.810551 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc65d4-12a2-442f-9a8e-15f338c6dd1b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:35.830013 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc65d4-12a2-442f-9a8e-15f338c6dd1b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:35.843594 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c62eab-2612-4c0f-87f9-f5583ff8e5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:35.844744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c62eab-2612-4c0f-87f9-f5583ff8e5cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:35.845030 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c62eab-2612-4c0f-87f9-f5583ff8e5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:35.846958 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c62eab-2612-4c0f-87f9-f5583ff8e5cd admin admin] PATH_INFO: `/v3/projects/112e3d3debdb429d8ee43ab4ac75466d/users/84f545e0ed6346d2a8e42111edca5e0b/roles/54195e660fc349978b39d5794765a095` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:35.896661 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c62eab-2612-4c0f-87f9-f5583ff8e5cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=112e3d3debdb429d8ee43ab4ac75466d, user_id=84f545e0ed6346d2a8e42111edca5e0b, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:35.913204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c62eab-2612-4c0f-87f9-f5583ff8e5cd admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:35.952538 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 826/1668] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:04:35 2022] PUT /identity/v3/projects/112e3d3debdb429d8ee43ab4ac75466d/users/84f545e0ed6346d2a8e42111edca5e0b/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:35.968347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b62a4-4b41-446c-bcf5-322ac935921b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:35.968797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b62a4-4b41-446c-bcf5-322ac935921b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:35.969220 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b62a4-4b41-446c-bcf5-322ac935921b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:36.021656 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b16b62a4-4b41-446c-bcf5-322ac935921b None None] MFA Rules not processed for user `84f545e0ed6346d2a8e42111edca5e0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:36.023929 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b16b62a4-4b41-446c-bcf5-322ac935921b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:36.061036 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435d5ac-f0dd-4949-b498-33481b994a43 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:36.061415 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435d5ac-f0dd-4949-b498-33481b994a43 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:36.061658 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435d5ac-f0dd-4949-b498-33481b994a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:36.116052 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:36.133056 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8435d5ac-f0dd-4949-b498-33481b994a43 None None] MFA Rules not processed for user `05afc537455c4a79be9fc8c4fdb1fc96`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:36.134872 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8435d5ac-f0dd-4949-b498-33481b994a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:36.137291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 843/1669] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:35 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:36.141127 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b62a4-4b41-446c-bcf5-322ac935921b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:36.251004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b62a4-4b41-446c-bcf5-322ac935921b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:36.256910 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc7b64-bec4-4d7f-99cf-eec57df906ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:36.257617 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc7b64-bec4-4d7f-99cf-eec57df906ea admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:36.257905 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc7b64-bec4-4d7f-99cf-eec57df906ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:36.258185 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc7b64-bec4-4d7f-99cf-eec57df906ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:36.259450 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc7b64-bec4-4d7f-99cf-eec57df906ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:36.262243 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc7b64-bec4-4d7f-99cf-eec57df906ea admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:36.294144 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 827/1670] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:04:36.307560 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377072c-f58b-425d-96e0-dc8d08d4b02c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:36.307857 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377072c-f58b-425d-96e0-dc8d08d4b02c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:36.308147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377072c-f58b-425d-96e0-dc8d08d4b02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:36.315713 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 844/1671] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:04:36 2022] POST /identity/v3/projects => generated 365 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:36.319242 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc7b64-bec4-4d7f-99cf-eec57df906ea admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:36.329898 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6377072c-f58b-425d-96e0-dc8d08d4b02c None None] MFA Rules not processed for user `05afc537455c4a79be9fc8c4fdb1fc96`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:36.331981 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6377072c-f58b-425d-96e0-dc8d08d4b02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:36.385707 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:36.402033 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 828/1672] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:36.415796 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377072c-f58b-425d-96e0-dc8d08d4b02c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:36.417908 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6377072c-f58b-425d-96e0-dc8d08d4b02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:36.465706 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc7b64-bec4-4d7f-99cf-eec57df906ea admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:36.474156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f26d2-53a9-494b-8779-db41272451e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:36.475437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f26d2-53a9-494b-8779-db41272451e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:36.475907 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f26d2-53a9-494b-8779-db41272451e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:36.476328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f26d2-53a9-494b-8779-db41272451e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:36.477212 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70f26d2-53a9-494b-8779-db41272451e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:36.480253 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70f26d2-53a9-494b-8779-db41272451e3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:36.503473 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377072c-f58b-425d-96e0-dc8d08d4b02c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:36.511562 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d497a857-bde5-49a0-badc-2a0fb8f98a94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:36.511764 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d497a857-bde5-49a0-badc-2a0fb8f98a94 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:36.511902 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d497a857-bde5-49a0-badc-2a0fb8f98a94 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:36.512180 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d497a857-bde5-49a0-badc-2a0fb8f98a94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:36.535198 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 845/1673] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:04:36 2022] POST /identity/v3/users => generated 289 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:04:36.536105 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d497a857-bde5-49a0-badc-2a0fb8f98a94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:36.536648 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f26d2-53a9-494b-8779-db41272451e3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:36.541048 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d497a857-bde5-49a0-badc-2a0fb8f98a94 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:36.559219 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f26d2-53a9-494b-8779-db41272451e3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:36.569072 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 829/1674] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:04:36.575761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce521d7-a598-48e5-8563-51386beb5682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:36.576782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce521d7-a598-48e5-8563-51386beb5682 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:36.576874 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce521d7-a598-48e5-8563-51386beb5682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:36.577157 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce521d7-a598-48e5-8563-51386beb5682 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:36.578367 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce521d7-a598-48e5-8563-51386beb5682 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:36.580310 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce521d7-a598-48e5-8563-51386beb5682 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:36.599024 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d497a857-bde5-49a0-badc-2a0fb8f98a94 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:36.606378 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 846/1675] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:36.615672 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d497a857-bde5-49a0-badc-2a0fb8f98a94 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:36.625253 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71049d-4106-448b-9701-e3eb4bebc7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:36.626522 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71049d-4106-448b-9701-e3eb4bebc7a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:36.626720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71049d-4106-448b-9701-e3eb4bebc7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:36.627009 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71049d-4106-448b-9701-e3eb4bebc7a0 admin admin] PATH_INFO: `/v3/projects/11b4b3cc43254958b0cc3ff7378a8019/users/b12616f91c17457ea71cdd43ad329222/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:36.679890 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a71049d-4106-448b-9701-e3eb4bebc7a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11b4b3cc43254958b0cc3ff7378a8019, user_id=b12616f91c17457ea71cdd43ad329222, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:36.692502 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a71049d-4106-448b-9701-e3eb4bebc7a0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:36.728757 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 830/1676] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:04:36 2022] PUT /identity/v3/projects/11b4b3cc43254958b0cc3ff7378a8019/users/b12616f91c17457ea71cdd43ad329222/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:04:36.731589 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce521d7-a598-48e5-8563-51386beb5682 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:36.744820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce521d7-a598-48e5-8563-51386beb5682 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:36.858546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:36.870423 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3572474e-8dc0-402b-9312-76da12796bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:36.871248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572474e-8dc0-402b-9312-76da12796bdf admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:36.873450 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572474e-8dc0-402b-9312-76da12796bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:36.873450 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572474e-8dc0-402b-9312-76da12796bdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:36.873450 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3572474e-8dc0-402b-9312-76da12796bdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:36.877272 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3572474e-8dc0-402b-9312-76da12796bdf admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:36.898737 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 847/1677] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:36.903038 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71049d-4106-448b-9701-e3eb4bebc7a0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:36.914173 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71049d-4106-448b-9701-e3eb4bebc7a0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:36.929653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792739ce-5b83-4b90-9955-eb6cd0cf25fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:36.931411 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792739ce-5b83-4b90-9955-eb6cd0cf25fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:36.931904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792739ce-5b83-4b90-9955-eb6cd0cf25fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:36.932533 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792739ce-5b83-4b90-9955-eb6cd0cf25fb admin admin] PATH_INFO: `/v3/projects/11b4b3cc43254958b0cc3ff7378a8019/users/b12616f91c17457ea71cdd43ad329222/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:36.934947 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792739ce-5b83-4b90-9955-eb6cd0cf25fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=11b4b3cc43254958b0cc3ff7378a8019, user_id=b12616f91c17457ea71cdd43ad329222, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:36.943798 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792739ce-5b83-4b90-9955-eb6cd0cf25fb admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:36.970932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 831/1678] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:04:36 2022] PUT /identity/v3/projects/11b4b3cc43254958b0cc3ff7378a8019/users/b12616f91c17457ea71cdd43ad329222/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:36.985229 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c1c9c-6a95-48dc-868d-804c5df9f2f1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:36.985538 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c1c9c-6a95-48dc-868d-804c5df9f2f1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:36.985922 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c1c9c-6a95-48dc-868d-804c5df9f2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:37.042112 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3e4c1c9c-6a95-48dc-868d-804c5df9f2f1 None None] MFA Rules not processed for user `b12616f91c17457ea71cdd43ad329222`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:37.043801 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3e4c1c9c-6a95-48dc-868d-804c5df9f2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:37.166281 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 848/1679] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:36 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:37.188932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792739ce-5b83-4b90-9955-eb6cd0cf25fb admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:37.190725 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-792739ce-5b83-4b90-9955-eb6cd0cf25fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:37.327665 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792739ce-5b83-4b90-9955-eb6cd0cf25fb admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:37.337867 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee7006c-6763-409c-9782-c8778e8ecd1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:37.338984 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7006c-6763-409c-9782-c8778e8ecd1e service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:37.339456 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7006c-6763-409c-9782-c8778e8ecd1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:37.339893 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7006c-6763-409c-9782-c8778e8ecd1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:37.356542 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee7006c-6763-409c-9782-c8778e8ecd1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:37.368868 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee7006c-6763-409c-9782-c8778e8ecd1e service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:37.389770 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 832/1680] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:37 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:04:37.527777 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c1c9c-6a95-48dc-868d-804c5df9f2f1 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:37.540266 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c1c9c-6a95-48dc-868d-804c5df9f2f1 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:37.551000 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b18bf0-4ed8-4e01-9575-b37127eb3b13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:37.551682 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b18bf0-4ed8-4e01-9575-b37127eb3b13 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:37.552078 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b18bf0-4ed8-4e01-9575-b37127eb3b13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:37.552245 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b18bf0-4ed8-4e01-9575-b37127eb3b13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:37.734502 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b18bf0-4ed8-4e01-9575-b37127eb3b13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:37.735990 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b18bf0-4ed8-4e01-9575-b37127eb3b13 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:37.772404 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 849/1681] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:37 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:04:38.172165 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090179ee-f8e6-44b4-a6ff-f4e933c9913a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:38.172463 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090179ee-f8e6-44b4-a6ff-f4e933c9913a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:38.172746 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090179ee-f8e6-44b4-a6ff-f4e933c9913a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:38.177957 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb417f-7181-487d-93ca-5c83fd60e4c6 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:38.178398 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb417f-7181-487d-93ca-5c83fd60e4c6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:38.178690 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb417f-7181-487d-93ca-5c83fd60e4c6 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:38.180594 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 833/1682] 10.222.0.52 () {54 vars in 912 bytes} [Thu Jul 21 16:04:38 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 16:04:38.205425 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b3cfe-999e-423b-a6d9-63e3948d305b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:38.205827 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b3cfe-999e-423b-a6d9-63e3948d305b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:38.206111 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b3cfe-999e-423b-a6d9-63e3948d305b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:38.238828 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-090179ee-f8e6-44b4-a6ff-f4e933c9913a None None] MFA Rules not processed for user `3f984e42c1d3464f94ea9693a9eeb55a`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:38.240202 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-090179ee-f8e6-44b4-a6ff-f4e933c9913a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:38.299094 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a19b3cfe-999e-423b-a6d9-63e3948d305b None None] MFA Rules not processed for user `1654fdf5ba6b46ba8d9b86a83a76a4b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:38.300345 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a19b3cfe-999e-423b-a6d9-63e3948d305b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:38.416746 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:38.478743 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 850/1683] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:38.494962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090179ee-f8e6-44b4-a6ff-f4e933c9913a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:38.499145 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-090179ee-f8e6-44b4-a6ff-f4e933c9913a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:38.519458 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 834/1684] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 21 16:04:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 21 16:04:38.535921 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b318498-478e-46a6-bd1f-3b1652566476 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:38.538029 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b318498-478e-46a6-bd1f-3b1652566476 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:38.538384 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b318498-478e-46a6-bd1f-3b1652566476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:38.544209 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:38.601736 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5b318498-478e-46a6-bd1f-3b1652566476 None None] MFA Rules not processed for user `a50cbf92286c420f9bee6e05c33dc440`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:38.604573 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5b318498-478e-46a6-bd1f-3b1652566476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:38.746037 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 835/1685] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:38.747007 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b318498-478e-46a6-bd1f-3b1652566476 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:38.749567 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5b318498-478e-46a6-bd1f-3b1652566476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:38.755801 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090179ee-f8e6-44b4-a6ff-f4e933c9913a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:38.769097 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b318498-478e-46a6-bd1f-3b1652566476 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:38.773421 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d015cc-7f8f-4157-8947-2269f92e8697 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:38.774969 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d015cc-7f8f-4157-8947-2269f92e8697 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:38.775414 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d015cc-7f8f-4157-8947-2269f92e8697 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:38.775964 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d015cc-7f8f-4157-8947-2269f92e8697 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:38.778074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4892d2f2-dca9-4f4e-b109-77d5ba2a18a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:38.778763 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892d2f2-dca9-4f4e-b109-77d5ba2a18a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:38.779071 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892d2f2-dca9-4f4e-b109-77d5ba2a18a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:38.779328 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892d2f2-dca9-4f4e-b109-77d5ba2a18a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:38.787564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d015cc-7f8f-4157-8947-2269f92e8697 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:38.788141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4892d2f2-dca9-4f4e-b109-77d5ba2a18a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:38.790532 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4892d2f2-dca9-4f4e-b109-77d5ba2a18a6 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:38.790948 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d015cc-7f8f-4157-8947-2269f92e8697 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:38.802706 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 851/1686] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:04:38.813803 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 836/1687] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:04:39.840653 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef72102-1fff-499e-83e5-2f3d95a7a362 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:39.841104 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef72102-1fff-499e-83e5-2f3d95a7a362 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:39.841417 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef72102-1fff-499e-83e5-2f3d95a7a362 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:39.843136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 852/1688] 10.222.0.52 () {54 vars in 961 bytes} [Thu Jul 21 16:04:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 16:04:39.852707 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336dc741-14f0-4389-87d7-5e3c377b8e02 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:39.853232 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336dc741-14f0-4389-87d7-5e3c377b8e02 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:39.853483 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336dc741-14f0-4389-87d7-5e3c377b8e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:39.903509 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-336dc741-14f0-4389-87d7-5e3c377b8e02 None None] MFA Rules not processed for user `822d9d3521db4a9688f51c6313c7dd74`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:39.906541 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-336dc741-14f0-4389-87d7-5e3c377b8e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:39.929643 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503cd34-a2e6-4410-b09b-c63e70c6847d None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:39.930311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503cd34-a2e6-4410-b09b-c63e70c6847d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:39.931306 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503cd34-a2e6-4410-b09b-c63e70c6847d None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:39.933811 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 853/1689] 10.222.0.52 () {54 vars in 912 bytes} [Thu Jul 21 16:04:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 16:04:39.961074 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99527dd1-977b-4644-b1fe-49232eb11cf8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:39.961968 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99527dd1-977b-4644-b1fe-49232eb11cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:39.962468 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99527dd1-977b-4644-b1fe-49232eb11cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:40.018171 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-99527dd1-977b-4644-b1fe-49232eb11cf8 None None] MFA Rules not processed for user `1654fdf5ba6b46ba8d9b86a83a76a4b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:40.028311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-99527dd1-977b-4644-b1fe-49232eb11cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:40.050555 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 854/1690] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 21 16:04:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:04:40.109437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99527dd1-977b-4644-b1fe-49232eb11cf8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:40.121697 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99527dd1-977b-4644-b1fe-49232eb11cf8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:40.137888 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a39805-3754-44cb-837d-60d8b62630cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:40.139183 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a39805-3754-44cb-837d-60d8b62630cc service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:40.142077 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a39805-3754-44cb-837d-60d8b62630cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:40.143251 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a39805-3754-44cb-837d-60d8b62630cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:40.146630 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 837/1691] 10.222.0.52 () {58 vars in 1058 bytes} [Thu Jul 21 16:04:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:40.157790 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336dc741-14f0-4389-87d7-5e3c377b8e02 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:40.172689 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336dc741-14f0-4389-87d7-5e3c377b8e02 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:40.179428 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a39805-3754-44cb-837d-60d8b62630cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:40.181219 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a39805-3754-44cb-837d-60d8b62630cc service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:40.192856 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be941c-c30d-4ea2-a112-5afee9feefff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822d9d3521db4a9688f51c6313c7dd74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:40.193536 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be941c-c30d-4ea2-a112-5afee9feefff service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:40.193847 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be941c-c30d-4ea2-a112-5afee9feefff service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:40.199009 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be941c-c30d-4ea2-a112-5afee9feefff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:40.212468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d4be941c-c30d-4ea2-a112-5afee9feefff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:40.223725 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 855/1692] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 16:04:40.252442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4be941c-c30d-4ea2-a112-5afee9feefff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:40.260411 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4be941c-c30d-4ea2-a112-5afee9feefff service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:40.301180 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 838/1693] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:04:42.546408 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a5fcd-630a-4b17-8251-a6f9a4d63222 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:42.546919 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a5fcd-630a-4b17-8251-a6f9a4d63222 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:42.547027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a5fcd-630a-4b17-8251-a6f9a4d63222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:42.611062 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fd0a5fcd-630a-4b17-8251-a6f9a4d63222 None None] MFA Rules not processed for user `3f984e42c1d3464f94ea9693a9eeb55a`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:42.618253 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fd0a5fcd-630a-4b17-8251-a6f9a4d63222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:42.667248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 856/1694] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:43.042057 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be941c-c30d-4ea2-a112-5afee9feefff service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:43.153362 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a5fcd-630a-4b17-8251-a6f9a4d63222 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:43.155142 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fd0a5fcd-630a-4b17-8251-a6f9a4d63222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:43.290743 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:43.303116 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be941c-c30d-4ea2-a112-5afee9feefff service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:43.323085 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eebd62-2afe-45b8-a9f9-0f74b5bf8115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:43.324564 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eebd62-2afe-45b8-a9f9-0f74b5bf8115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:43.324856 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eebd62-2afe-45b8-a9f9-0f74b5bf8115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:43.325131 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eebd62-2afe-45b8-a9f9-0f74b5bf8115 admin admin] PATH_INFO: `/v3/users/84f545e0ed6346d2a8e42111edca5e0b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:43.330932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eebd62-2afe-45b8-a9f9-0f74b5bf8115 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84f545e0ed6346d2a8e42111edca5e0b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:43.336167 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eebd62-2afe-45b8-a9f9-0f74b5bf8115 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:43.366953 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a5fcd-630a-4b17-8251-a6f9a4d63222 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:43.373466 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731fe2e1-294f-42a4-a790-ddf8e6ffacb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:43.375119 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731fe2e1-294f-42a4-a790-ddf8e6ffacb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:43.375789 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731fe2e1-294f-42a4-a790-ddf8e6ffacb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:43.376225 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731fe2e1-294f-42a4-a790-ddf8e6ffacb1 admin admin] PATH_INFO: `/v3/users/761b320197dc46a4a517cf7e56276c09` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:43.379050 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731fe2e1-294f-42a4-a790-ddf8e6ffacb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=761b320197dc46a4a517cf7e56276c09)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:43.381527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731fe2e1-294f-42a4-a790-ddf8e6ffacb1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:43.436513 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c8eebd62-2afe-45b8-a9f9-0f74b5bf8115 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84f545e0ed6346d2a8e42111edca5e0b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:43.438629 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c8eebd62-2afe-45b8-a9f9-0f74b5bf8115 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84f545e0ed6346d2a8e42111edca5e0b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:43.439516 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c8eebd62-2afe-45b8-a9f9-0f74b5bf8115 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84f545e0ed6346d2a8e42111edca5e0b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:43.467804 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c8eebd62-2afe-45b8-a9f9-0f74b5bf8115 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84f545e0ed6346d2a8e42111edca5e0b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:43.496116 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c8eebd62-2afe-45b8-a9f9-0f74b5bf8115 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84f545e0ed6346d2a8e42111edca5e0b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:43.510341 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-731fe2e1-294f-42a4-a790-ddf8e6ffacb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '761b320197dc46a4a517cf7e56276c09'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:43.511393 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-731fe2e1-294f-42a4-a790-ddf8e6ffacb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '761b320197dc46a4a517cf7e56276c09'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:43.512147 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-731fe2e1-294f-42a4-a790-ddf8e6ffacb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '761b320197dc46a4a517cf7e56276c09'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:43.525828 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-731fe2e1-294f-42a4-a790-ddf8e6ffacb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '761b320197dc46a4a517cf7e56276c09'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:43.571632 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 839/1695] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:04:43 2022] DELETE /identity/v3/users/84f545e0ed6346d2a8e42111edca5e0b => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:43.575569 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-731fe2e1-294f-42a4-a790-ddf8e6ffacb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '761b320197dc46a4a517cf7e56276c09'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:43.577257 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30986da4-e48d-4991-bfca-3f2acf3a07cc None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:43.577657 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30986da4-e48d-4991-bfca-3f2acf3a07cc None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:43.577964 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30986da4-e48d-4991-bfca-3f2acf3a07cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:43.620554 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:43.621122 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-30986da4-e48d-4991-bfca-3f2acf3a07cc None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:43.628440 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-30986da4-e48d-4991-bfca-3f2acf3a07cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:43.686608 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 857/1696] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:04:43 2022] DELETE /identity/v3/users/761b320197dc46a4a517cf7e56276c09 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:43.691972 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50044a-eef4-435e-a8da-13133d7a76c7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:43.693462 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50044a-eef4-435e-a8da-13133d7a76c7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:43.693905 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50044a-eef4-435e-a8da-13133d7a76c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:43.737735 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3b50044a-eef4-435e-a8da-13133d7a76c7 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:43.739420 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3b50044a-eef4-435e-a8da-13133d7a76c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:43.870733 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 840/1697] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:43.946475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 858/1698] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:43.989458 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30986da4-e48d-4991-bfca-3f2acf3a07cc None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:43.991546 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-30986da4-e48d-4991-bfca-3f2acf3a07cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:44.048895 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50044a-eef4-435e-a8da-13133d7a76c7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:44.050917 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3b50044a-eef4-435e-a8da-13133d7a76c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:44.257922 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30986da4-e48d-4991-bfca-3f2acf3a07cc None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:44.277076 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80117fb2-79ea-4013-8b07-b4f6d05b099a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:44.278072 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80117fb2-79ea-4013-8b07-b4f6d05b099a service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:44.279853 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80117fb2-79ea-4013-8b07-b4f6d05b099a service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:44.280364 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80117fb2-79ea-4013-8b07-b4f6d05b099a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:44.293785 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50044a-eef4-435e-a8da-13133d7a76c7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:44.305652 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57034b-e93c-4d13-8971-25422026f839 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:44.309436 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57034b-e93c-4d13-8971-25422026f839 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:44.311185 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57034b-e93c-4d13-8971-25422026f839 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:44.314036 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57034b-e93c-4d13-8971-25422026f839 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:44.315738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80117fb2-79ea-4013-8b07-b4f6d05b099a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:44.322218 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80117fb2-79ea-4013-8b07-b4f6d05b099a service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:44.361108 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be57034b-e93c-4d13-8971-25422026f839 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:44.362688 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be57034b-e93c-4d13-8971-25422026f839 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:44.376820 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 841/1699] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:04:44.412172 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 859/1700] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:04:44.756392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a4e11-e12b-4a88-8f6d-21ae5a199b12 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:44.757115 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a4e11-e12b-4a88-8f6d-21ae5a199b12 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:44.757366 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a4e11-e12b-4a88-8f6d-21ae5a199b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:44.837930 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:44.893355 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9f3a4e11-e12b-4a88-8f6d-21ae5a199b12 None None] MFA Rules not processed for user `6edbad98ed0f4823b9955f9f0e270149`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:44.895645 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9f3a4e11-e12b-4a88-8f6d-21ae5a199b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:45.103344 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 842/1701] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 21 16:04:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:04:45.125366 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57034b-e93c-4d13-8971-25422026f839 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:45.126895 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-be57034b-e93c-4d13-8971-25422026f839 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:45.146978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57034b-e93c-4d13-8971-25422026f839 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:45.159056 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c0950-5f4a-4bac-a0aa-1030f90c484d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:45.159974 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c0950-5f4a-4bac-a0aa-1030f90c484d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:45.160459 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c0950-5f4a-4bac-a0aa-1030f90c484d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:45.160899 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c0950-5f4a-4bac-a0aa-1030f90c484d admin admin] PATH_INFO: `/v3/users/734d0448ba5c4c0e86ab1feb6c871fb8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:45.162646 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c0950-5f4a-4bac-a0aa-1030f90c484d admin admin] RBAC: Authorizing `identity:delete_user(user_id=734d0448ba5c4c0e86ab1feb6c871fb8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:45.167778 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c0950-5f4a-4bac-a0aa-1030f90c484d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:45.320363 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-729c0950-5f4a-4bac-a0aa-1030f90c484d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '734d0448ba5c4c0e86ab1feb6c871fb8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:45.322220 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-729c0950-5f4a-4bac-a0aa-1030f90c484d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '734d0448ba5c4c0e86ab1feb6c871fb8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:45.323123 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-729c0950-5f4a-4bac-a0aa-1030f90c484d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '734d0448ba5c4c0e86ab1feb6c871fb8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:45.363365 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-729c0950-5f4a-4bac-a0aa-1030f90c484d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '734d0448ba5c4c0e86ab1feb6c871fb8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:45.380328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-729c0950-5f4a-4bac-a0aa-1030f90c484d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '734d0448ba5c4c0e86ab1feb6c871fb8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:45.423695 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:45.439425 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 860/1702] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:04:45 2022] DELETE /identity/v3/users/734d0448ba5c4c0e86ab1feb6c871fb8 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:45.444200 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ce319-4ed5-4537-8772-61a3599e7d1f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:45.444679 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ce319-4ed5-4537-8772-61a3599e7d1f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:45.446327 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ce319-4ed5-4537-8772-61a3599e7d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:45.462732 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c0950-5f4a-4bac-a0aa-1030f90c484d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:45.468587 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-729c0950-5f4a-4bac-a0aa-1030f90c484d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:45.487863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7f5ce319-4ed5-4537-8772-61a3599e7d1f None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:45.489118 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7f5ce319-4ed5-4537-8772-61a3599e7d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:45.649035 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 843/1703] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:45.706978 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ce319-4ed5-4537-8772-61a3599e7d1f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:45.708653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7f5ce319-4ed5-4537-8772-61a3599e7d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:45.717778 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c0950-5f4a-4bac-a0aa-1030f90c484d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:45.732679 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d749f3d-1f29-4280-9acd-423496416655 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5bfae2cb30843b1b1a0343efef38638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:45.735448 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d749f3d-1f29-4280-9acd-423496416655 service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:45.735863 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d749f3d-1f29-4280-9acd-423496416655 service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:45.737788 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d749f3d-1f29-4280-9acd-423496416655 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:45.743898 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3d749f3d-1f29-4280-9acd-423496416655 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:45.842727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ce319-4ed5-4537-8772-61a3599e7d1f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:45.854485 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07b2919-6ac2-4664-bf67-dd94aa23c2cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:45.855522 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b2919-6ac2-4664-bf67-dd94aa23c2cd service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:45.855819 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b2919-6ac2-4664-bf67-dd94aa23c2cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:45.856110 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b2919-6ac2-4664-bf67-dd94aa23c2cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:45.881499 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07b2919-6ac2-4664-bf67-dd94aa23c2cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:45.890677 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d749f3d-1f29-4280-9acd-423496416655 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:45.892432 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d749f3d-1f29-4280-9acd-423496416655 service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:45.895431 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07b2919-6ac2-4664-bf67-dd94aa23c2cd service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:45.919020 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 844/1704] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 16:04:45.925143 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 861/1705] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 16:04:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 16:04:46.074623 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07b2919-6ac2-4664-bf67-dd94aa23c2cd service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:46.076424 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d07b2919-6ac2-4664-bf67-dd94aa23c2cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:46.114279 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07b2919-6ac2-4664-bf67-dd94aa23c2cd service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:46.129417 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b131c1-7547-4d55-8113-7027f09e6382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:46.130616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b131c1-7547-4d55-8113-7027f09e6382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:46.131085 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b131c1-7547-4d55-8113-7027f09e6382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:46.131516 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b131c1-7547-4d55-8113-7027f09e6382 admin admin] PATH_INFO: `/v3/projects/d6b449a0c00f40b7898872ceabe812cf` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:46.133783 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b131c1-7547-4d55-8113-7027f09e6382 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6b449a0c00f40b7898872ceabe812cf)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:46.147813 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b131c1-7547-4d55-8113-7027f09e6382 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:46.345442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:46.403108 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-65b131c1-7547-4d55-8113-7027f09e6382 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6b449a0c00f40b7898872ceabe812cf', 'request_id': 'req-65b131c1-7547-4d55-8113-7027f09e6382'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:46.422204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-65b131c1-7547-4d55-8113-7027f09e6382 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6b449a0c00f40b7898872ceabe812cf', 'request_id': 'req-65b131c1-7547-4d55-8113-7027f09e6382'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:46.445547 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-65b131c1-7547-4d55-8113-7027f09e6382 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6b449a0c00f40b7898872ceabe812cf', 'request_id': 'req-65b131c1-7547-4d55-8113-7027f09e6382'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:46.489539 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-65b131c1-7547-4d55-8113-7027f09e6382 admin admin] The token cache is being invalidate because project d6b449a0c00f40b7898872ceabe812cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:04:46.492247 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-65b131c1-7547-4d55-8113-7027f09e6382 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:46.498446 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-65b131c1-7547-4d55-8113-7027f09e6382 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:46.506298 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 845/1706] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:04:46 2022] DELETE /identity/v3/projects/d6b449a0c00f40b7898872ceabe812cf => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:04:46.510515 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b926d-65d8-42d3-8b20-2c15b04fcade None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:46.510983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b926d-65d8-42d3-8b20-2c15b04fcade None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:46.511250 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b926d-65d8-42d3-8b20-2c15b04fcade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:46.561483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f68b926d-65d8-42d3-8b20-2c15b04fcade None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:46.565476 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f68b926d-65d8-42d3-8b20-2c15b04fcade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:46.735648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d081362-a946-48c5-bc1b-984f22b9d849 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:46.736198 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d081362-a946-48c5-bc1b-984f22b9d849 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:46.736684 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d081362-a946-48c5-bc1b-984f22b9d849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:46.747220 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 862/1707] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:46.771945 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6d081362-a946-48c5-bc1b-984f22b9d849 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:46.775304 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6d081362-a946-48c5-bc1b-984f22b9d849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:46.802015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b926d-65d8-42d3-8b20-2c15b04fcade None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:46.802716 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 846/1708] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:46.813929 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b926d-65d8-42d3-8b20-2c15b04fcade None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:46.844182 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fdb3cc-10ab-413a-9fab-696b9ab5e96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:46.844850 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fdb3cc-10ab-413a-9fab-696b9ab5e96f admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:46.845133 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fdb3cc-10ab-413a-9fab-696b9ab5e96f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:46.845598 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fdb3cc-10ab-413a-9fab-696b9ab5e96f admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:46.846496 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fdb3cc-10ab-413a-9fab-696b9ab5e96f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:04:46.846971 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fdb3cc-10ab-413a-9fab-696b9ab5e96f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:46.849138 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fdb3cc-10ab-413a-9fab-696b9ab5e96f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:46.894824 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 863/1709] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:04:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:04:46.896236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d081362-a946-48c5-bc1b-984f22b9d849 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:46.912798 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d081362-a946-48c5-bc1b-984f22b9d849 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:46.925271 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a4f9ee-2ac5-45dc-a37d-be10ec41d24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:46.929180 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4f9ee-2ac5-45dc-a37d-be10ec41d24f admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:46.930914 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4f9ee-2ac5-45dc-a37d-be10ec41d24f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:46.932707 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4f9ee-2ac5-45dc-a37d-be10ec41d24f admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:46.933977 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a4f9ee-2ac5-45dc-a37d-be10ec41d24f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:46.939761 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a4f9ee-2ac5-45dc-a37d-be10ec41d24f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:47.034295 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 847/1710] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:04:46 2022] POST /identity/v3/projects => generated 371 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:04:47.035324 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fdb3cc-10ab-413a-9fab-696b9ab5e96f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:47.050996 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fdb3cc-10ab-413a-9fab-696b9ab5e96f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:47.072797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:47.231662 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949420fa-9bdb-46d7-8dd3-0c06b0c851fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:47.231662 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949420fa-9bdb-46d7-8dd3-0c06b0c851fd admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:47.231662 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949420fa-9bdb-46d7-8dd3-0c06b0c851fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:47.232036 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949420fa-9bdb-46d7-8dd3-0c06b0c851fd admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:47.232711 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949420fa-9bdb-46d7-8dd3-0c06b0c851fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:47.238648 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949420fa-9bdb-46d7-8dd3-0c06b0c851fd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:47.281191 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 864/1711] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:04:47 2022] POST /identity/v3/users => generated 286 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:47.283820 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a4f9ee-2ac5-45dc-a37d-be10ec41d24f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:47.303933 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a4f9ee-2ac5-45dc-a37d-be10ec41d24f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:47.317057 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f2da7-f804-4ad2-b2f2-b9ed0bbd13e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:47.319873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f2da7-f804-4ad2-b2f2-b9ed0bbd13e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:47.321344 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f2da7-f804-4ad2-b2f2-b9ed0bbd13e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:47.322194 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f2da7-f804-4ad2-b2f2-b9ed0bbd13e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:47.323255 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f2da7-f804-4ad2-b2f2-b9ed0bbd13e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:47.332547 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f2da7-f804-4ad2-b2f2-b9ed0bbd13e2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:47.381202 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 848/1712] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:47.385539 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949420fa-9bdb-46d7-8dd3-0c06b0c851fd admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:47.406357 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949420fa-9bdb-46d7-8dd3-0c06b0c851fd admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:47.421353 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47d78c1-1bd9-4efd-962e-06180edc4b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:47.423381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d78c1-1bd9-4efd-962e-06180edc4b9f admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:47.425389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d78c1-1bd9-4efd-962e-06180edc4b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:47.425957 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d78c1-1bd9-4efd-962e-06180edc4b9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:47.427043 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47d78c1-1bd9-4efd-962e-06180edc4b9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:47.431858 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47d78c1-1bd9-4efd-962e-06180edc4b9f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:47.464405 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 865/1713] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:47.470306 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f2da7-f804-4ad2-b2f2-b9ed0bbd13e2 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:47.504054 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f2da7-f804-4ad2-b2f2-b9ed0bbd13e2 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:47.534089 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ccf6d0-8242-43bf-8267-e7a5842d314a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:47.548424 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccf6d0-8242-43bf-8267-e7a5842d314a admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:47.549102 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccf6d0-8242-43bf-8267-e7a5842d314a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:47.549595 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccf6d0-8242-43bf-8267-e7a5842d314a admin admin] PATH_INFO: `/v3/projects/cef8be2cb85e49b687bfd50b22a9a091/users/270cc4487383469ca8db5e7489a1c614/roles/54195e660fc349978b39d5794765a095` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:47.625279 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ccf6d0-8242-43bf-8267-e7a5842d314a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cef8be2cb85e49b687bfd50b22a9a091, user_id=270cc4487383469ca8db5e7489a1c614, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:47.634435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ccf6d0-8242-43bf-8267-e7a5842d314a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:47.667142 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 849/1714] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:04:47 2022] PUT /identity/v3/projects/cef8be2cb85e49b687bfd50b22a9a091/users/270cc4487383469ca8db5e7489a1c614/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:47.677106 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b878d5a-c637-462a-ad76-f66737c10a8a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:47.677776 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b878d5a-c637-462a-ad76-f66737c10a8a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:47.677776 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b878d5a-c637-462a-ad76-f66737c10a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:47.714112 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf698ee4-3598-4427-81b8-0bf80b07b7e0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:47.714851 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf698ee4-3598-4427-81b8-0bf80b07b7e0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:47.716234 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf698ee4-3598-4427-81b8-0bf80b07b7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:47.760630 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-bf698ee4-3598-4427-81b8-0bf80b07b7e0 None None] MFA Rules not processed for user `1654fdf5ba6b46ba8d9b86a83a76a4b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:47.762718 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bf698ee4-3598-4427-81b8-0bf80b07b7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:47.768379 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9b878d5a-c637-462a-ad76-f66737c10a8a None None] MFA Rules not processed for user `270cc4487383469ca8db5e7489a1c614`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:47.770258 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9b878d5a-c637-462a-ad76-f66737c10a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:48.067038 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 866/1715] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:48.067948 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b878d5a-c637-462a-ad76-f66737c10a8a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:48.072040 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9b878d5a-c637-462a-ad76-f66737c10a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:48.093071 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 850/1716] 10.222.0.66 () {58 vars in 1015 bytes} [Thu Jul 21 16:04:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:48.094005 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf698ee4-3598-4427-81b8-0bf80b07b7e0 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:48.099568 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bf698ee4-3598-4427-81b8-0bf80b07b7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:48.265834 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b878d5a-c637-462a-ad76-f66737c10a8a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:48.282376 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b122a-7323-44a6-95cd-1d3945f950fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:48.283517 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b122a-7323-44a6-95cd-1d3945f950fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:48.283947 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b122a-7323-44a6-95cd-1d3945f950fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:48.284378 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b122a-7323-44a6-95cd-1d3945f950fd admin admin] PATH_INFO: `/v3/projects/112e3d3debdb429d8ee43ab4ac75466d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:48.286545 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b122a-7323-44a6-95cd-1d3945f950fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=112e3d3debdb429d8ee43ab4ac75466d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:48.288630 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b122a-7323-44a6-95cd-1d3945f950fd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:48.319304 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:48.373303 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-835b122a-7323-44a6-95cd-1d3945f950fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '112e3d3debdb429d8ee43ab4ac75466d', 'request_id': 'req-835b122a-7323-44a6-95cd-1d3945f950fd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:48.391071 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-835b122a-7323-44a6-95cd-1d3945f950fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '112e3d3debdb429d8ee43ab4ac75466d', 'request_id': 'req-835b122a-7323-44a6-95cd-1d3945f950fd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:48.413615 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-835b122a-7323-44a6-95cd-1d3945f950fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '112e3d3debdb429d8ee43ab4ac75466d', 'request_id': 'req-835b122a-7323-44a6-95cd-1d3945f950fd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:48.469116 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-835b122a-7323-44a6-95cd-1d3945f950fd admin admin] The token cache is being invalidate because project 112e3d3debdb429d8ee43ab4ac75466d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:04:48.474303 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-835b122a-7323-44a6-95cd-1d3945f950fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:48.482252 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-835b122a-7323-44a6-95cd-1d3945f950fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:48.483493 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 867/1717] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:04:48 2022] DELETE /identity/v3/projects/112e3d3debdb429d8ee43ab4ac75466d => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:48.487854 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b122a-7323-44a6-95cd-1d3945f950fd admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:48.490102 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-835b122a-7323-44a6-95cd-1d3945f950fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:48.493337 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:48.510648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf698ee4-3598-4427-81b8-0bf80b07b7e0 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:48.749227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee74ad1-cf16-4ec2-bad5-b00f33c063cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:48.752030 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee74ad1-cf16-4ec2-bad5-b00f33c063cb service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:48.752752 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee74ad1-cf16-4ec2-bad5-b00f33c063cb service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:48.753721 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee74ad1-cf16-4ec2-bad5-b00f33c063cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:48.759371 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-dee74ad1-cf16-4ec2-bad5-b00f33c063cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:48.760729 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b122a-7323-44a6-95cd-1d3945f950fd admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:48.780068 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2681d-f2af-418e-9f27-bb3676b83e87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:48.780750 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2681d-f2af-418e-9f27-bb3676b83e87 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:48.781035 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2681d-f2af-418e-9f27-bb3676b83e87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:48.781315 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2681d-f2af-418e-9f27-bb3676b83e87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:48.784927 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ebf2681d-f2af-418e-9f27-bb3676b83e87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:48.808946 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf2681d-f2af-418e-9f27-bb3676b83e87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:48.810465 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf2681d-f2af-418e-9f27-bb3676b83e87 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:48.835310 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 868/1718] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:04:48.836955 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2681d-f2af-418e-9f27-bb3676b83e87 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:48.842078 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ebf2681d-f2af-418e-9f27-bb3676b83e87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:48.958241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee74ad1-cf16-4ec2-bad5-b00f33c063cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:48.962946 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee74ad1-cf16-4ec2-bad5-b00f33c063cb service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:48.986673 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 851/1719] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 893 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:04:48.988534 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee74ad1-cf16-4ec2-bad5-b00f33c063cb service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:48.996036 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-dee74ad1-cf16-4ec2-bad5-b00f33c063cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:49.092132 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2681d-f2af-418e-9f27-bb3676b83e87 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:49.101300 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eced991-4ec2-4f6f-8ca2-f2844f3b7058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:49.102407 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eced991-4ec2-4f6f-8ca2-f2844f3b7058 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:49.102702 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eced991-4ec2-4f6f-8ca2-f2844f3b7058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:49.103031 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eced991-4ec2-4f6f-8ca2-f2844f3b7058 admin admin] PATH_INFO: `/v3/projects/2f8a894e68d04bf589a9b42b6a78e64e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:49.106171 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eced991-4ec2-4f6f-8ca2-f2844f3b7058 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f8a894e68d04bf589a9b42b6a78e64e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:49.108149 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eced991-4ec2-4f6f-8ca2-f2844f3b7058 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:49.205269 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1eced991-4ec2-4f6f-8ca2-f2844f3b7058 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f8a894e68d04bf589a9b42b6a78e64e', 'request_id': 'req-1eced991-4ec2-4f6f-8ca2-f2844f3b7058'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:49.212727 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:49.212923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee74ad1-cf16-4ec2-bad5-b00f33c063cb service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:49.224625 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1eced991-4ec2-4f6f-8ca2-f2844f3b7058 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f8a894e68d04bf589a9b42b6a78e64e', 'request_id': 'req-1eced991-4ec2-4f6f-8ca2-f2844f3b7058'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:49.283177 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1eced991-4ec2-4f6f-8ca2-f2844f3b7058 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f8a894e68d04bf589a9b42b6a78e64e', 'request_id': 'req-1eced991-4ec2-4f6f-8ca2-f2844f3b7058'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:49.310702 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1eced991-4ec2-4f6f-8ca2-f2844f3b7058 admin admin] The token cache is being invalidate because project 2f8a894e68d04bf589a9b42b6a78e64e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:04:49.311088 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1eced991-4ec2-4f6f-8ca2-f2844f3b7058 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:49.314027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1eced991-4ec2-4f6f-8ca2-f2844f3b7058 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:49.314181 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:49.316716 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 869/1720] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:04:48 2022] DELETE /identity/v3/projects/2f8a894e68d04bf589a9b42b6a78e64e => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:49.327550 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eced991-4ec2-4f6f-8ca2-f2844f3b7058 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:49.331568 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1eced991-4ec2-4f6f-8ca2-f2844f3b7058 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:49.354154 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4744af-4c2b-485c-8b07-9fc77356a538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:49.356415 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4744af-4c2b-485c-8b07-9fc77356a538 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:49.356924 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4744af-4c2b-485c-8b07-9fc77356a538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:49.358524 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4744af-4c2b-485c-8b07-9fc77356a538 admin admin] PATH_INFO: `/v3/users/35805c9448d848628d872a94462b1d6b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:49.369042 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4744af-4c2b-485c-8b07-9fc77356a538 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35805c9448d848628d872a94462b1d6b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:49.372767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eced991-4ec2-4f6f-8ca2-f2844f3b7058 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:49.374475 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4744af-4c2b-485c-8b07-9fc77356a538 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:49.405744 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77105f-c2e9-4a65-bdf0-ef6b17830e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:49.411477 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77105f-c2e9-4a65-bdf0-ef6b17830e64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:49.411477 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77105f-c2e9-4a65-bdf0-ef6b17830e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:49.416631 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77105f-c2e9-4a65-bdf0-ef6b17830e64 admin admin] PATH_INFO: `/v3/users/b12616f91c17457ea71cdd43ad329222` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:49.420965 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a77105f-c2e9-4a65-bdf0-ef6b17830e64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b12616f91c17457ea71cdd43ad329222)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:49.432489 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a77105f-c2e9-4a65-bdf0-ef6b17830e64 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:49.573654 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0a77105f-c2e9-4a65-bdf0-ef6b17830e64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b12616f91c17457ea71cdd43ad329222'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:49.576012 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0a77105f-c2e9-4a65-bdf0-ef6b17830e64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b12616f91c17457ea71cdd43ad329222'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:49.577118 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0a77105f-c2e9-4a65-bdf0-ef6b17830e64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b12616f91c17457ea71cdd43ad329222'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:49.577538 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7c4744af-4c2b-485c-8b07-9fc77356a538 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35805c9448d848628d872a94462b1d6b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:49.578419 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7c4744af-4c2b-485c-8b07-9fc77356a538 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35805c9448d848628d872a94462b1d6b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:49.579299 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7c4744af-4c2b-485c-8b07-9fc77356a538 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35805c9448d848628d872a94462b1d6b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:49.599917 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0a77105f-c2e9-4a65-bdf0-ef6b17830e64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b12616f91c17457ea71cdd43ad329222'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:49.606473 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7c4744af-4c2b-485c-8b07-9fc77356a538 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35805c9448d848628d872a94462b1d6b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:49.619802 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0a77105f-c2e9-4a65-bdf0-ef6b17830e64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b12616f91c17457ea71cdd43ad329222'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:49.626109 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7c4744af-4c2b-485c-8b07-9fc77356a538 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35805c9448d848628d872a94462b1d6b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:49.661306 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 870/1721] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:04:49 2022] DELETE /identity/v3/users/b12616f91c17457ea71cdd43ad329222 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:49.661306 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77105f-c2e9-4a65-bdf0-ef6b17830e64 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:49.668555 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0a77105f-c2e9-4a65-bdf0-ef6b17830e64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:49.674587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 852/1722] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:04:48 2022] DELETE /identity/v3/users/35805c9448d848628d872a94462b1d6b => generated 0 bytes in 687 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:49.680191 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728bbb52-5446-4bce-bd23-902cdfc16e40 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:49.681981 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728bbb52-5446-4bce-bd23-902cdfc16e40 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:49.682390 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728bbb52-5446-4bce-bd23-902cdfc16e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:49.742652 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-728bbb52-5446-4bce-bd23-902cdfc16e40 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:49.755255 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-728bbb52-5446-4bce-bd23-902cdfc16e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:49.844133 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77105f-c2e9-4a65-bdf0-ef6b17830e64 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:49.963036 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 853/1723] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:49.990370 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013c65d-a6a6-4d8a-8426-add8c402b623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:49.991270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013c65d-a6a6-4d8a-8426-add8c402b623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:49.991558 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013c65d-a6a6-4d8a-8426-add8c402b623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:49.991839 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013c65d-a6a6-4d8a-8426-add8c402b623 admin admin] PATH_INFO: `/v3/users/9bdc10d2d386413b841833f72fa557da` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:49.994327 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e013c65d-a6a6-4d8a-8426-add8c402b623 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bdc10d2d386413b841833f72fa557da)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:49.996255 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e013c65d-a6a6-4d8a-8426-add8c402b623 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:50.005526 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728bbb52-5446-4bce-bd23-902cdfc16e40 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:50.007204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-728bbb52-5446-4bce-bd23-902cdfc16e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:50.014994 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:50.171706 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e013c65d-a6a6-4d8a-8426-add8c402b623 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bdc10d2d386413b841833f72fa557da'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:50.172459 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e013c65d-a6a6-4d8a-8426-add8c402b623 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bdc10d2d386413b841833f72fa557da'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:50.173129 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e013c65d-a6a6-4d8a-8426-add8c402b623 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bdc10d2d386413b841833f72fa557da'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:50.194802 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e013c65d-a6a6-4d8a-8426-add8c402b623 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bdc10d2d386413b841833f72fa557da'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:50.210660 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e013c65d-a6a6-4d8a-8426-add8c402b623 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bdc10d2d386413b841833f72fa557da'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:50.252585 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728bbb52-5446-4bce-bd23-902cdfc16e40 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:50.254761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 871/1724] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:04:49 2022] DELETE /identity/v3/users/9bdc10d2d386413b841833f72fa557da => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:50.316542 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:50.419755 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e17b9-887b-4148-9c6f-5a6576bba396 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:50.420440 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e17b9-887b-4148-9c6f-5a6576bba396 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:50.420730 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e17b9-887b-4148-9c6f-5a6576bba396 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:50.426313 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e17b9-887b-4148-9c6f-5a6576bba396 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:50.433525 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b74e17b9-887b-4148-9c6f-5a6576bba396 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:50.722876 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e17b9-887b-4148-9c6f-5a6576bba396 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:50.725446 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e17b9-887b-4148-9c6f-5a6576bba396 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:50.750781 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 854/1725] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 746 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:04:52.755273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013c65d-a6a6-4d8a-8426-add8c402b623 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:52.757011 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e013c65d-a6a6-4d8a-8426-add8c402b623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:52.817537 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013c65d-a6a6-4d8a-8426-add8c402b623 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:52.885038 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa78c3e-aa46-455c-8697-e01731e64fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:52.886359 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa78c3e-aa46-455c-8697-e01731e64fd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:52.887280 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa78c3e-aa46-455c-8697-e01731e64fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:52.887768 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa78c3e-aa46-455c-8697-e01731e64fd1 admin admin] PATH_INFO: `/v3/projects/5bb03f8c149f4135adbaabc2102e6d59` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:52.892488 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa78c3e-aa46-455c-8697-e01731e64fd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bb03f8c149f4135adbaabc2102e6d59)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:52.898360 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa78c3e-aa46-455c-8697-e01731e64fd1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:52.925332 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e17b9-887b-4148-9c6f-5a6576bba396 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:52.930897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b74e17b9-887b-4148-9c6f-5a6576bba396 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:52.977863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e17b9-887b-4148-9c6f-5a6576bba396 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:53.024832 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9fa78c3e-aa46-455c-8697-e01731e64fd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bb03f8c149f4135adbaabc2102e6d59', 'request_id': 'req-9fa78c3e-aa46-455c-8697-e01731e64fd1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:53.042252 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9fa78c3e-aa46-455c-8697-e01731e64fd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bb03f8c149f4135adbaabc2102e6d59', 'request_id': 'req-9fa78c3e-aa46-455c-8697-e01731e64fd1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:53.049411 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9fa78c3e-aa46-455c-8697-e01731e64fd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bb03f8c149f4135adbaabc2102e6d59', 'request_id': 'req-9fa78c3e-aa46-455c-8697-e01731e64fd1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:53.080367 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9fa78c3e-aa46-455c-8697-e01731e64fd1 admin admin] The token cache is being invalidate because project 5bb03f8c149f4135adbaabc2102e6d59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:04:53.080367 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9fa78c3e-aa46-455c-8697-e01731e64fd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:53.083979 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9fa78c3e-aa46-455c-8697-e01731e64fd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:53.094902 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 872/1726] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:04:52 2022] DELETE /identity/v3/projects/5bb03f8c149f4135adbaabc2102e6d59 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:53.094902 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa78c3e-aa46-455c-8697-e01731e64fd1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:53.096635 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c84bc7-551e-40e6-9623-ef3aa346aaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:53.098704 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9fa78c3e-aa46-455c-8697-e01731e64fd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:53.099673 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c84bc7-551e-40e6-9623-ef3aa346aaff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:53.100059 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c84bc7-551e-40e6-9623-ef3aa346aaff admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:53.100415 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c84bc7-551e-40e6-9623-ef3aa346aaff admin admin] PATH_INFO: `/v3/projects/e57e443306ff4510b03a45d02e27a033` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:53.103409 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c84bc7-551e-40e6-9623-ef3aa346aaff admin admin] RBAC: Authorizing `identity:delete_project(project_id=e57e443306ff4510b03a45d02e27a033)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:53.107637 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c84bc7-551e-40e6-9623-ef3aa346aaff admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:53.143473 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa78c3e-aa46-455c-8697-e01731e64fd1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:53.157045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e70296a-d6d5-4b4a-9d9b-99c70a368114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:53.158213 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70296a-d6d5-4b4a-9d9b-99c70a368114 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:53.158596 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70296a-d6d5-4b4a-9d9b-99c70a368114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:53.158974 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70296a-d6d5-4b4a-9d9b-99c70a368114 admin admin] PATH_INFO: `/v3/projects/11b4b3cc43254958b0cc3ff7378a8019` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:53.160458 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e70296a-d6d5-4b4a-9d9b-99c70a368114 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11b4b3cc43254958b0cc3ff7378a8019)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:53.165059 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e70296a-d6d5-4b4a-9d9b-99c70a368114 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:53.184476 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-62c84bc7-551e-40e6-9623-ef3aa346aaff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e57e443306ff4510b03a45d02e27a033', 'request_id': 'req-62c84bc7-551e-40e6-9623-ef3aa346aaff'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:53.200174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-62c84bc7-551e-40e6-9623-ef3aa346aaff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e57e443306ff4510b03a45d02e27a033', 'request_id': 'req-62c84bc7-551e-40e6-9623-ef3aa346aaff'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:53.213280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-62c84bc7-551e-40e6-9623-ef3aa346aaff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e57e443306ff4510b03a45d02e27a033', 'request_id': 'req-62c84bc7-551e-40e6-9623-ef3aa346aaff'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:53.263292 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-62c84bc7-551e-40e6-9623-ef3aa346aaff admin admin] The token cache is being invalidate because project e57e443306ff4510b03a45d02e27a033 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:04:53.265011 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-62c84bc7-551e-40e6-9623-ef3aa346aaff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:53.265761 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-62c84bc7-551e-40e6-9623-ef3aa346aaff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:53.269438 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 855/1727] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:04:52 2022] DELETE /identity/v3/projects/e57e443306ff4510b03a45d02e27a033 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:53.272969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9231d-2911-40a0-9aaa-a1967d594f2d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:53.274413 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9231d-2911-40a0-9aaa-a1967d594f2d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:53.274726 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9231d-2911-40a0-9aaa-a1967d594f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:53.280362 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4e70296a-d6d5-4b4a-9d9b-99c70a368114 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11b4b3cc43254958b0cc3ff7378a8019', 'request_id': 'req-4e70296a-d6d5-4b4a-9d9b-99c70a368114'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:53.288653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:53.295891 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4e70296a-d6d5-4b4a-9d9b-99c70a368114 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11b4b3cc43254958b0cc3ff7378a8019', 'request_id': 'req-4e70296a-d6d5-4b4a-9d9b-99c70a368114'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:53.297794 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=75,keystone:DELETE=22,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:53.308398 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4e70296a-d6d5-4b4a-9d9b-99c70a368114 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11b4b3cc43254958b0cc3ff7378a8019', 'request_id': 'req-4e70296a-d6d5-4b4a-9d9b-99c70a368114'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:53.321302 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-40f9231d-2911-40a0-9aaa-a1967d594f2d None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:53.324856 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-40f9231d-2911-40a0-9aaa-a1967d594f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:53.332587 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4e70296a-d6d5-4b4a-9d9b-99c70a368114 admin admin] The token cache is being invalidate because project 11b4b3cc43254958b0cc3ff7378a8019 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:04:53.332949 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4e70296a-d6d5-4b4a-9d9b-99c70a368114 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:53.334683 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4e70296a-d6d5-4b4a-9d9b-99c70a368114 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:53.342299 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 873/1728] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:04:53 2022] DELETE /identity/v3/projects/11b4b3cc43254958b0cc3ff7378a8019 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:53.345395 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d17e1-8983-4b7b-ae32-0ffb9c0bd143 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:53.345395 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d17e1-8983-4b7b-ae32-0ffb9c0bd143 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:53.345800 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d17e1-8983-4b7b-ae32-0ffb9c0bd143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:53.382144 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1c8d17e1-8983-4b7b-ae32-0ffb9c0bd143 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:53.382403 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1c8d17e1-8983-4b7b-ae32-0ffb9c0bd143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:53.527942 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 874/1729] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:53.529215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91a0f6-01a1-4285-b062-9a3ef6d317ea None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:53.529505 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91a0f6-01a1-4285-b062-9a3ef6d317ea None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:53.532335 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91a0f6-01a1-4285-b062-9a3ef6d317ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:53.567432 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 856/1730] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:53.569337 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6879c7d5-cc46-4eb4-a147-94275d9574a2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:53.569865 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6879c7d5-cc46-4eb4-a147-94275d9574a2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:53.570198 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6879c7d5-cc46-4eb4-a147-94275d9574a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:53.573720 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2f91a0f6-01a1-4285-b062-9a3ef6d317ea None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:53.578288 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2f91a0f6-01a1-4285-b062-9a3ef6d317ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:53.613652 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6879c7d5-cc46-4eb4-a147-94275d9574a2 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:53.628028 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6879c7d5-cc46-4eb4-a147-94275d9574a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:53.635589 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 875/1731] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:53.640807 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79f5e5-f2f3-452b-9a19-1b2e7708a7fa None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:53.641655 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79f5e5-f2f3-452b-9a19-1b2e7708a7fa None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:53.642438 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79f5e5-f2f3-452b-9a19-1b2e7708a7fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:53.650401 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 857/1732] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:53.655989 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4488a-7258-4833-b8d4-f90cbc0ca68a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:53.656365 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4488a-7258-4833-b8d4-f90cbc0ca68a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:53.656667 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4488a-7258-4833-b8d4-f90cbc0ca68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:53.708659 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f4f4488a-7258-4833-b8d4-f90cbc0ca68a None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:53.709938 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f4f4488a-7258-4833-b8d4-f90cbc0ca68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:53.714568 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-af79f5e5-f2f3-452b-9a19-1b2e7708a7fa None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:53.719427 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-af79f5e5-f2f3-452b-9a19-1b2e7708a7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:53.741201 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 858/1733] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:53.754533 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4488a-7258-4833-b8d4-f90cbc0ca68a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:53.755179 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 876/1734] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:53.759042 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af79f5e5-f2f3-452b-9a19-1b2e7708a7fa None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:53.784688 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4488a-7258-4833-b8d4-f90cbc0ca68a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:53.790117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af79f5e5-f2f3-452b-9a19-1b2e7708a7fa None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:53.818472 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048fde99-256b-4528-89e0-9de88e71c72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:53.829217 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048fde99-256b-4528-89e0-9de88e71c72b admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:53.829217 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048fde99-256b-4528-89e0-9de88e71c72b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:53.829217 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048fde99-256b-4528-89e0-9de88e71c72b admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:53.829217 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048fde99-256b-4528-89e0-9de88e71c72b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:04:53.829217 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048fde99-256b-4528-89e0-9de88e71c72b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:53.829799 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24525526-af08-4da6-b78f-f66df463c9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:53.829799 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24525526-af08-4da6-b78f-f66df463c9ef admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:53.829799 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24525526-af08-4da6-b78f-f66df463c9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:53.831790 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24525526-af08-4da6-b78f-f66df463c9ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:53.831912 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048fde99-256b-4528-89e0-9de88e71c72b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:53.832465 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24525526-af08-4da6-b78f-f66df463c9ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:04:53.832887 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24525526-af08-4da6-b78f-f66df463c9ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:53.838875 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24525526-af08-4da6-b78f-f66df463c9ef admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:53.884246 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 877/1735] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:04:53.888081 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 859/1736] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:04:53.888081 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24525526-af08-4da6-b78f-f66df463c9ef admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:53.888335 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048fde99-256b-4528-89e0-9de88e71c72b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:53.924595 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048fde99-256b-4528-89e0-9de88e71c72b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:53.928162 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24525526-af08-4da6-b78f-f66df463c9ef admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:53.941977 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5912cd5-edf5-40cb-a441-bcafdc1d6544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:53.942659 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5912cd5-edf5-40cb-a441-bcafdc1d6544 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:53.942950 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5912cd5-edf5-40cb-a441-bcafdc1d6544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:53.943232 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5912cd5-edf5-40cb-a441-bcafdc1d6544 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:53.943950 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5912cd5-edf5-40cb-a441-bcafdc1d6544 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:53.949372 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad528f-f17a-4845-9da9-f528047b8fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:53.950372 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad528f-f17a-4845-9da9-f528047b8fcd admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:53.950828 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad528f-f17a-4845-9da9-f528047b8fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:53.951268 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad528f-f17a-4845-9da9-f528047b8fcd admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:53.952460 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad528f-f17a-4845-9da9-f528047b8fcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:04:53.952942 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad528f-f17a-4845-9da9-f528047b8fcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:53.954940 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad528f-f17a-4845-9da9-f528047b8fcd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:53.959925 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5912cd5-edf5-40cb-a441-bcafdc1d6544 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:53.973364 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 878/1737] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:04:53.974792 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad528f-f17a-4845-9da9-f528047b8fcd admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:53.995701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad528f-f17a-4845-9da9-f528047b8fcd admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:54.031861 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb07ca-88ec-4dde-adf0-0bb1af895ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:54.033099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb07ca-88ec-4dde-adf0-0bb1af895ccc admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:54.033913 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb07ca-88ec-4dde-adf0-0bb1af895ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:54.034742 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb07ca-88ec-4dde-adf0-0bb1af895ccc admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:54.035673 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb07ca-88ec-4dde-adf0-0bb1af895ccc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:54.040016 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb07ca-88ec-4dde-adf0-0bb1af895ccc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:54.127066 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 879/1738] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:04:53 2022] POST /identity/v3/projects => generated 373 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:54.135905 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 860/1739] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:04:53 2022] POST /identity/v3/projects => generated 367 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:54.137306 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fd546-4c33-4fc8-a623-6a10a025514d None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:54.141067 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb07ca-88ec-4dde-adf0-0bb1af895ccc admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:54.147600 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fd546-4c33-4fc8-a623-6a10a025514d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:54.147895 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fd546-4c33-4fc8-a623-6a10a025514d None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:54.154638 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 861/1740] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jul 21 16:04:54 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 16:04:54.157849 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9fd546-4c33-4fc8-a623-6a10a025514d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:54.164826 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7a9fd546-4c33-4fc8-a623-6a10a025514d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:54.301014 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb07ca-88ec-4dde-adf0-0bb1af895ccc admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:54.420179 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:54.433011 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9fd546-4c33-4fc8-a623-6a10a025514d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:54.446445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb7224c-2069-4044-bfcd-1f1925ca6031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:54.447567 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb7224c-2069-4044-bfcd-1f1925ca6031 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:54.447853 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb7224c-2069-4044-bfcd-1f1925ca6031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:54.448185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb7224c-2069-4044-bfcd-1f1925ca6031 admin admin] PATH_INFO: `/v3/users/270cc4487383469ca8db5e7489a1c614` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:54.454116 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb7224c-2069-4044-bfcd-1f1925ca6031 admin admin] RBAC: Authorizing `identity:delete_user(user_id=270cc4487383469ca8db5e7489a1c614)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:54.456372 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb7224c-2069-4044-bfcd-1f1925ca6031 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:54.502872 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b2306-5033-43f4-9c4e-a319288936a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:54.503745 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b2306-5033-43f4-9c4e-a319288936a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:54.504177 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b2306-5033-43f4-9c4e-a319288936a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:54.504624 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b2306-5033-43f4-9c4e-a319288936a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:54.505451 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b2306-5033-43f4-9c4e-a319288936a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:54.515227 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b2306-5033-43f4-9c4e-a319288936a4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:54.598514 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8eb7224c-2069-4044-bfcd-1f1925ca6031 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '270cc4487383469ca8db5e7489a1c614'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:54.599807 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8eb7224c-2069-4044-bfcd-1f1925ca6031 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '270cc4487383469ca8db5e7489a1c614'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:54.601076 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8eb7224c-2069-4044-bfcd-1f1925ca6031 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '270cc4487383469ca8db5e7489a1c614'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:54.605832 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 880/1741] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:04:54 2022] POST /identity/v3/projects => generated 369 bytes in 478 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:54.606696 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b2306-5033-43f4-9c4e-a319288936a4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:54.611766 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-798b2306-5033-43f4-9c4e-a319288936a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:54.626476 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8eb7224c-2069-4044-bfcd-1f1925ca6031 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '270cc4487383469ca8db5e7489a1c614'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:54.629512 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:54.650779 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8eb7224c-2069-4044-bfcd-1f1925ca6031 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '270cc4487383469ca8db5e7489a1c614'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:54.727123 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 862/1742] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:04:54 2022] DELETE /identity/v3/users/270cc4487383469ca8db5e7489a1c614 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:54.732247 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb7224c-2069-4044-bfcd-1f1925ca6031 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:54.738400 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8eb7224c-2069-4044-bfcd-1f1925ca6031 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:54.797130 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b2306-5033-43f4-9c4e-a319288936a4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:54.930786 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04014b0a-1502-4915-a70a-540eba9d183b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:54.931618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04014b0a-1502-4915-a70a-540eba9d183b admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:54.932061 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04014b0a-1502-4915-a70a-540eba9d183b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:54.932382 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04014b0a-1502-4915-a70a-540eba9d183b admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:54.933188 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04014b0a-1502-4915-a70a-540eba9d183b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:54.935545 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04014b0a-1502-4915-a70a-540eba9d183b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:54.982298 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 881/1743] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:04:54 2022] POST /identity/v3/users => generated 287 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:54.982298 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a239b9-11e1-42a1-90cb-f451d079c36a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:54.982298 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a239b9-11e1-42a1-90cb-f451d079c36a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:54.982298 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a239b9-11e1-42a1-90cb-f451d079c36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:55.046192 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb7224c-2069-4044-bfcd-1f1925ca6031 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:55.051155 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-97a239b9-11e1-42a1-90cb-f451d079c36a None None] MFA Rules not processed for user `092553a86f8848dba8c7271f5c0b2c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:55.059298 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65947cf-e774-4408-8679-fd2c0084483c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:55.060366 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65947cf-e774-4408-8679-fd2c0084483c admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:55.060715 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65947cf-e774-4408-8679-fd2c0084483c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:55.061138 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65947cf-e774-4408-8679-fd2c0084483c admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:55.061971 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65947cf-e774-4408-8679-fd2c0084483c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:55.062216 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-97a239b9-11e1-42a1-90cb-f451d079c36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:55.064145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65947cf-e774-4408-8679-fd2c0084483c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:55.111547 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 863/1744] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:04:54 2022] POST /identity/v3/users => generated 284 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:55.114496 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65947cf-e774-4408-8679-fd2c0084483c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:55.123392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d65947cf-e774-4408-8679-fd2c0084483c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:55.149459 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65947cf-e774-4408-8679-fd2c0084483c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:55.167563 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b17f03-d667-443d-a9c8-9259c74ff832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:55.168466 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b17f03-d667-443d-a9c8-9259c74ff832 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:55.168819 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b17f03-d667-443d-a9c8-9259c74ff832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:55.169166 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b17f03-d667-443d-a9c8-9259c74ff832 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:55.170617 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b17f03-d667-443d-a9c8-9259c74ff832 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:55.178321 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b17f03-d667-443d-a9c8-9259c74ff832 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:55.204656 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:55.230300 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 864/1745] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:04:55 2022] POST /identity/v3/users => generated 285 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:55.234177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567343bb-5918-425c-b9db-86e98c8c0d17 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:55.234574 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567343bb-5918-425c-b9db-86e98c8c0d17 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:55.234936 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567343bb-5918-425c-b9db-86e98c8c0d17 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:55.241741 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 865/1746] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jul 21 16:04:55 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 16:04:55.243640 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcfd89-d36b-4ca6-bf63-89e7e8ea8462 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:55.243928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcfd89-d36b-4ca6-bf63-89e7e8ea8462 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:55.244206 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcfd89-d36b-4ca6-bf63-89e7e8ea8462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:55.252348 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 882/1747] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jul 21 16:04:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:55.253889 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a239b9-11e1-42a1-90cb-f451d079c36a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:55.269233 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a239b9-11e1-42a1-90cb-f451d079c36a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:55.276600 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665171b-f154-46bb-9e6f-f939cfc82bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:55.277641 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665171b-f154-46bb-9e6f-f939cfc82bf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:55.278133 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665171b-f154-46bb-9e6f-f939cfc82bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:55.278686 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665171b-f154-46bb-9e6f-f939cfc82bf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:55.279696 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665171b-f154-46bb-9e6f-f939cfc82bf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:55.282042 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665171b-f154-46bb-9e6f-f939cfc82bf8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:55.286706 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-23bcfd89-d36b-4ca6-bf63-89e7e8ea8462 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:55.288188 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-23bcfd89-d36b-4ca6-bf63-89e7e8ea8462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:55.300676 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 883/1748] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:55.302051 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665171b-f154-46bb-9e6f-f939cfc82bf8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:55.312433 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 866/1749] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:55.314858 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcfd89-d36b-4ca6-bf63-89e7e8ea8462 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:55.324276 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665171b-f154-46bb-9e6f-f939cfc82bf8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:55.341329 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcfd89-d36b-4ca6-bf63-89e7e8ea8462 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:55.342735 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e5b2f5-9860-4297-a3e3-3a9f6bf25370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:55.343735 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5b2f5-9860-4297-a3e3-3a9f6bf25370 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:55.344302 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5b2f5-9860-4297-a3e3-3a9f6bf25370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:55.344786 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5b2f5-9860-4297-a3e3-3a9f6bf25370 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:55.349167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e5b2f5-9860-4297-a3e3-3a9f6bf25370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:55.354941 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ff651-9d22-41fc-84cc-db86a9a5af81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:55.355186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e5b2f5-9860-4297-a3e3-3a9f6bf25370 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:55.355847 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ff651-9d22-41fc-84cc-db86a9a5af81 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:55.356208 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ff651-9d22-41fc-84cc-db86a9a5af81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:55.356552 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ff651-9d22-41fc-84cc-db86a9a5af81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:55.357436 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ff651-9d22-41fc-84cc-db86a9a5af81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:55.359438 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ff651-9d22-41fc-84cc-db86a9a5af81 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:55.384756 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 884/1750] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:55.386783 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e5b2f5-9860-4297-a3e3-3a9f6bf25370 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:55.404723 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e5b2f5-9860-4297-a3e3-3a9f6bf25370 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:55.413220 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f60f3c-6039-4b13-8403-5c8fa3189d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:55.415755 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f60f3c-6039-4b13-8403-5c8fa3189d7c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:55.416993 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 867/1751] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:55.418088 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f60f3c-6039-4b13-8403-5c8fa3189d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:55.418630 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed120d13-fa5b-4e42-bb27-0d39ca4aeac0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:55.418911 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed120d13-fa5b-4e42-bb27-0d39ca4aeac0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:55.419172 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f60f3c-6039-4b13-8403-5c8fa3189d7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:55.419287 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed120d13-fa5b-4e42-bb27-0d39ca4aeac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:55.421072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f60f3c-6039-4b13-8403-5c8fa3189d7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:55.425725 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f60f3c-6039-4b13-8403-5c8fa3189d7c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:55.448418 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 885/1752] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:55.452145 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f60f3c-6039-4b13-8403-5c8fa3189d7c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:55.466947 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ed120d13-fa5b-4e42-bb27-0d39ca4aeac0 None None] MFA Rules not processed for user `092553a86f8848dba8c7271f5c0b2c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:55.468699 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ed120d13-fa5b-4e42-bb27-0d39ca4aeac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:55.491618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f60f3c-6039-4b13-8403-5c8fa3189d7c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:55.495209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 868/1753] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jul 21 16:04:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:55.496355 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed120d13-fa5b-4e42-bb27-0d39ca4aeac0 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:55.501115 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711500b9-c105-404e-ac90-d40139b5b012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:55.501892 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ed120d13-fa5b-4e42-bb27-0d39ca4aeac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:55.504248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711500b9-c105-404e-ac90-d40139b5b012 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:55.505521 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711500b9-c105-404e-ac90-d40139b5b012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:55.506419 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711500b9-c105-404e-ac90-d40139b5b012 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:55.508987 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711500b9-c105-404e-ac90-d40139b5b012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:55.511982 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711500b9-c105-404e-ac90-d40139b5b012 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:55.535253 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 886/1754] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:55.538031 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711500b9-c105-404e-ac90-d40139b5b012 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:55.558830 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711500b9-c105-404e-ac90-d40139b5b012 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:55.568362 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61109a5c-317c-4904-ad63-3db843563dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:55.575253 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61109a5c-317c-4904-ad63-3db843563dcb admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:55.575253 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61109a5c-317c-4904-ad63-3db843563dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:55.575253 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61109a5c-317c-4904-ad63-3db843563dcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:55.575253 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61109a5c-317c-4904-ad63-3db843563dcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:55.600205 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61109a5c-317c-4904-ad63-3db843563dcb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:55.642399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 887/1755] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:55.645881 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61109a5c-317c-4904-ad63-3db843563dcb admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:55.678745 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed120d13-fa5b-4e42-bb27-0d39ca4aeac0 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:55.685268 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61109a5c-317c-4904-ad63-3db843563dcb admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:55.688013 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044af8bb-564c-4ac5-acf8-b64f0b21c6fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:55.689010 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044af8bb-564c-4ac5-acf8-b64f0b21c6fc service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:55.689538 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044af8bb-564c-4ac5-acf8-b64f0b21c6fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:55.692195 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044af8bb-564c-4ac5-acf8-b64f0b21c6fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:55.720006 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044af8bb-564c-4ac5-acf8-b64f0b21c6fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:55.721791 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a3a71-b2cf-49b6-a39a-772ddd333073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:55.723652 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044af8bb-564c-4ac5-acf8-b64f0b21c6fc service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:55.726863 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a3a71-b2cf-49b6-a39a-772ddd333073 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:55.728508 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a3a71-b2cf-49b6-a39a-772ddd333073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:55.730925 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a3a71-b2cf-49b6-a39a-772ddd333073 admin admin] PATH_INFO: `/v3/projects/37aea02c7b974071a39754881deb779a/users/1e61ec5021c84fdc849ee222f764e76b/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:55.758389 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 869/1756] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:04:55.759548 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044af8bb-564c-4ac5-acf8-b64f0b21c6fc service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:55.781619 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044af8bb-564c-4ac5-acf8-b64f0b21c6fc service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:55.782910 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9a3a71-b2cf-49b6-a39a-772ddd333073 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37aea02c7b974071a39754881deb779a, user_id=1e61ec5021c84fdc849ee222f764e76b, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:55.791608 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9a3a71-b2cf-49b6-a39a-772ddd333073 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:55.796865 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e90fe7-adbc-4073-a0c9-f76d19cb772e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:55.800564 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e90fe7-adbc-4073-a0c9-f76d19cb772e admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:55.801805 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e90fe7-adbc-4073-a0c9-f76d19cb772e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:55.802338 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e90fe7-adbc-4073-a0c9-f76d19cb772e admin admin] PATH_INFO: `/v3/projects/489be9911f7a4eb9be1e60a073e13bce/users/eacadfd6d7bf4fb28a69df8315113d6f/roles/54195e660fc349978b39d5794765a095` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:55.849379 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e90fe7-adbc-4073-a0c9-f76d19cb772e admin admin] RBAC: Authorizing `identity:create_grant(project_id=489be9911f7a4eb9be1e60a073e13bce, user_id=eacadfd6d7bf4fb28a69df8315113d6f, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:55.855296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 888/1757] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:04:55 2022] PUT /identity/v3/projects/37aea02c7b974071a39754881deb779a/users/1e61ec5021c84fdc849ee222f764e76b/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:55.859014 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a3a71-b2cf-49b6-a39a-772ddd333073 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:55.859573 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e90fe7-adbc-4073-a0c9-f76d19cb772e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:55.882151 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:INSERT=8,keystone:DELETE=3,keystone:SELECT=89 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:55.889462 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 870/1758] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:04:55 2022] PUT /identity/v3/projects/489be9911f7a4eb9be1e60a073e13bce/users/eacadfd6d7bf4fb28a69df8315113d6f/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:55.894861 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e90fe7-adbc-4073-a0c9-f76d19cb772e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:55.902116 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-16e90fe7-adbc-4073-a0c9-f76d19cb772e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:56.053887 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e90fe7-adbc-4073-a0c9-f76d19cb772e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:56.059988 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1397ab-b70b-4f1d-8f17-0b1ab07781b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:56.060662 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1397ab-b70b-4f1d-8f17-0b1ab07781b8 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:56.061429 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1397ab-b70b-4f1d-8f17-0b1ab07781b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:56.061429 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1397ab-b70b-4f1d-8f17-0b1ab07781b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:56.078056 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1397ab-b70b-4f1d-8f17-0b1ab07781b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:56.080902 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1397ab-b70b-4f1d-8f17-0b1ab07781b8 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:56.097476 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 871/1759] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:04:56.112145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe68c45e-7cbe-4211-ac4b-529448f8c137 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:56.112871 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe68c45e-7cbe-4211-ac4b-529448f8c137 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:56.114069 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a3a71-b2cf-49b6-a39a-772ddd333073 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:56.114721 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe68c45e-7cbe-4211-ac4b-529448f8c137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:56.192997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-fe68c45e-7cbe-4211-ac4b-529448f8c137 None None] MFA Rules not processed for user `1e61ec5021c84fdc849ee222f764e76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:56.200189 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fe68c45e-7cbe-4211-ac4b-529448f8c137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:56.263745 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a327b-f547-4249-95fb-ba1e9c6939bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:56.265325 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a327b-f547-4249-95fb-ba1e9c6939bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:56.265858 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a327b-f547-4249-95fb-ba1e9c6939bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:56.266442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a327b-f547-4249-95fb-ba1e9c6939bd admin admin] PATH_INFO: `/v3/projects/9fffbf9dacce4a6c8e7e807474c39cd7/users/d67f09592176445fbe84218772502f17/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:56.322859 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a327b-f547-4249-95fb-ba1e9c6939bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fffbf9dacce4a6c8e7e807474c39cd7, user_id=d67f09592176445fbe84218772502f17, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:56.326142 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a327b-f547-4249-95fb-ba1e9c6939bd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:56.331155 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:56.359917 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 889/1760] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:04:55 2022] PUT /identity/v3/projects/9fffbf9dacce4a6c8e7e807474c39cd7/users/d67f09592176445fbe84218772502f17/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:56.375483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b86119-d5d7-41fc-b50e-b91432a77e1f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:56.376317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b86119-d5d7-41fc-b50e-b91432a77e1f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:56.377124 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b86119-d5d7-41fc-b50e-b91432a77e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:56.403440 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 872/1761] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:56.404643 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe68c45e-7cbe-4211-ac4b-529448f8c137 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:56.434797 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe68c45e-7cbe-4211-ac4b-529448f8c137 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:56.440286 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-32b86119-d5d7-41fc-b50e-b91432a77e1f None None] MFA Rules not processed for user `eacadfd6d7bf4fb28a69df8315113d6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:56.442833 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-32b86119-d5d7-41fc-b50e-b91432a77e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:56.587095 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 890/1762] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:56 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:56.589017 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49432c-1b1d-4781-aa1b-f9c56e486441 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:56.589449 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49432c-1b1d-4781-aa1b-f9c56e486441 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:56.590115 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49432c-1b1d-4781-aa1b-f9c56e486441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:56.593032 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9724fe-9c5d-493e-8b90-b86f680e5709 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:56.594002 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9724fe-9c5d-493e-8b90-b86f680e5709 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:56.594504 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9724fe-9c5d-493e-8b90-b86f680e5709 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:56.594998 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9724fe-9c5d-493e-8b90-b86f680e5709 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:56.621036 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9724fe-9c5d-493e-8b90-b86f680e5709 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:56.623437 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9724fe-9c5d-493e-8b90-b86f680e5709 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:56.639566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-db49432c-1b1d-4781-aa1b-f9c56e486441 None None] MFA Rules not processed for user `d67f09592176445fbe84218772502f17`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:56.639958 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:56.641904 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-db49432c-1b1d-4781-aa1b-f9c56e486441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:56.652362 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 873/1763] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:04:56.662388 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9724fe-9c5d-493e-8b90-b86f680e5709 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:56.884078 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 891/1764] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:56 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:56.886676 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db49432c-1b1d-4781-aa1b-f9c56e486441 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:56.895186 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9724fe-9c5d-493e-8b90-b86f680e5709 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:56.919145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19798cf4-7202-4d6a-a027-fb8d00b4776f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:56.919936 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19798cf4-7202-4d6a-a027-fb8d00b4776f admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:56.920326 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19798cf4-7202-4d6a-a027-fb8d00b4776f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:56.920758 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19798cf4-7202-4d6a-a027-fb8d00b4776f admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:56.921651 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19798cf4-7202-4d6a-a027-fb8d00b4776f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:56.934090 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19798cf4-7202-4d6a-a027-fb8d00b4776f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:56.934718 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db49432c-1b1d-4781-aa1b-f9c56e486441 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:56.943464 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631393b5-064f-4307-846a-4c4edcfbeab5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:56.944146 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631393b5-064f-4307-846a-4c4edcfbeab5 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:56.944599 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631393b5-064f-4307-846a-4c4edcfbeab5 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:56.945987 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631393b5-064f-4307-846a-4c4edcfbeab5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:57.067351 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 874/1765] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:04:56 2022] POST /identity/v3/projects => generated 367 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:57.071637 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19798cf4-7202-4d6a-a027-fb8d00b4776f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:57.126022 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631393b5-064f-4307-846a-4c4edcfbeab5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:57.134307 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631393b5-064f-4307-846a-4c4edcfbeab5 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:57.182744 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 892/1766] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:04:57.186045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631393b5-064f-4307-846a-4c4edcfbeab5 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:57.243672 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19798cf4-7202-4d6a-a027-fb8d00b4776f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:57.245906 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631393b5-064f-4307-846a-4c4edcfbeab5 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:57.256294 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5266d-c29b-4b66-a074-17cca5df6a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:57.258888 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5266d-c29b-4b66-a074-17cca5df6a59 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:57.259329 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5266d-c29b-4b66-a074-17cca5df6a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:57.259862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5266d-c29b-4b66-a074-17cca5df6a59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:57.264875 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e5266d-c29b-4b66-a074-17cca5df6a59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:57.267319 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e5266d-c29b-4b66-a074-17cca5df6a59 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:57.270881 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24950d-b311-440b-979a-7cf14ec6958c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:57.273954 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24950d-b311-440b-979a-7cf14ec6958c admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:57.276828 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24950d-b311-440b-979a-7cf14ec6958c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:57.279892 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24950d-b311-440b-979a-7cf14ec6958c admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:57.283937 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c24950d-b311-440b-979a-7cf14ec6958c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:57.298426 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c24950d-b311-440b-979a-7cf14ec6958c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:57.319218 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 875/1767] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:04:57 2022] POST /identity/v3/projects => generated 365 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:57.323992 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5266d-c29b-4b66-a074-17cca5df6a59 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:57.331567 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:57.364849 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 893/1768] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:04:57 2022] POST /identity/v3/users => generated 290 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:57.366756 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24950d-b311-440b-979a-7cf14ec6958c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:57.391837 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24950d-b311-440b-979a-7cf14ec6958c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:57.489898 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49072169-42c6-4c1c-a1c1-967fd9ef7b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:57.490806 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49072169-42c6-4c1c-a1c1-967fd9ef7b5c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:57.491103 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49072169-42c6-4c1c-a1c1-967fd9ef7b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:57.491476 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49072169-42c6-4c1c-a1c1-967fd9ef7b5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:57.492302 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49072169-42c6-4c1c-a1c1-967fd9ef7b5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:57.494134 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49072169-42c6-4c1c-a1c1-967fd9ef7b5c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:57.510542 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 894/1769] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:57.515582 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49072169-42c6-4c1c-a1c1-967fd9ef7b5c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:57.537498 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49072169-42c6-4c1c-a1c1-967fd9ef7b5c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:57.549379 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8766633-859d-4624-a838-9e6b9f02f773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:57.551487 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8766633-859d-4624-a838-9e6b9f02f773 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:57.552712 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8766633-859d-4624-a838-9e6b9f02f773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:57.554245 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8766633-859d-4624-a838-9e6b9f02f773 admin admin] PATH_INFO: `/v3/projects/05b45b43b383499da00bff9eabb1f5a8/users/5d830eeb1bc44e5bbd4e160ad7003725/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:57.559037 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5266d-c29b-4b66-a074-17cca5df6a59 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:57.565821 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97615a81-eb61-4191-a0e1-94d4fac56142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:57.566681 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97615a81-eb61-4191-a0e1-94d4fac56142 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:57.567039 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97615a81-eb61-4191-a0e1-94d4fac56142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:57.567412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97615a81-eb61-4191-a0e1-94d4fac56142 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:57.568207 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97615a81-eb61-4191-a0e1-94d4fac56142 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:57.572175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97615a81-eb61-4191-a0e1-94d4fac56142 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:57.578031 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8766633-859d-4624-a838-9e6b9f02f773 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05b45b43b383499da00bff9eabb1f5a8, user_id=5d830eeb1bc44e5bbd4e160ad7003725, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:57.581201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8766633-859d-4624-a838-9e6b9f02f773 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:57.605312 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:57.610024 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 895/1770] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:04:57 2022] PUT /identity/v3/projects/05b45b43b383499da00bff9eabb1f5a8/users/5d830eeb1bc44e5bbd4e160ad7003725/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:57.617199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 876/1771] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:04:57 2022] POST /identity/v3/users => generated 289 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:57.621283 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8766633-859d-4624-a838-9e6b9f02f773 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:57.621720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97615a81-eb61-4191-a0e1-94d4fac56142 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:57.658368 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97615a81-eb61-4191-a0e1-94d4fac56142 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:57.784223 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8766633-859d-4624-a838-9e6b9f02f773 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:57.797907 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e95b16-6abe-4fe1-8e4b-ca75a95bf6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:57.798887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e95b16-6abe-4fe1-8e4b-ca75a95bf6b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:57.798981 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e95b16-6abe-4fe1-8e4b-ca75a95bf6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:57.799296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e95b16-6abe-4fe1-8e4b-ca75a95bf6b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:57.800120 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e95b16-6abe-4fe1-8e4b-ca75a95bf6b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:57.802997 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e95b16-6abe-4fe1-8e4b-ca75a95bf6b4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:57.836900 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 896/1772] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:57.846491 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec19f83-a378-4e49-95d0-607f5fa54548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:57.846980 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e95b16-6abe-4fe1-8e4b-ca75a95bf6b4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:57.847815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec19f83-a378-4e49-95d0-607f5fa54548 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:57.848664 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec19f83-a378-4e49-95d0-607f5fa54548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:57.849102 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec19f83-a378-4e49-95d0-607f5fa54548 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:57.850035 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec19f83-a378-4e49-95d0-607f5fa54548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:57.851986 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec19f83-a378-4e49-95d0-607f5fa54548 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:57.867834 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e95b16-6abe-4fe1-8e4b-ca75a95bf6b4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:57.878351 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 877/1773] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:57.879628 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec19f83-a378-4e49-95d0-607f5fa54548 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:57.880307 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99201ea5-0bb7-4e8f-97e8-4e6743593cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:57.881439 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99201ea5-0bb7-4e8f-97e8-4e6743593cee admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:57.882288 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99201ea5-0bb7-4e8f-97e8-4e6743593cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:57.883168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99201ea5-0bb7-4e8f-97e8-4e6743593cee admin admin] PATH_INFO: `/v3/projects/05b45b43b383499da00bff9eabb1f5a8/users/5d830eeb1bc44e5bbd4e160ad7003725/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:57.886037 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99201ea5-0bb7-4e8f-97e8-4e6743593cee admin admin] RBAC: Authorizing `identity:create_grant(project_id=05b45b43b383499da00bff9eabb1f5a8, user_id=5d830eeb1bc44e5bbd4e160ad7003725, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:57.893410 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec19f83-a378-4e49-95d0-607f5fa54548 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:57.901192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99201ea5-0bb7-4e8f-97e8-4e6743593cee admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:57.903053 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2895b76f-d486-4524-bd47-69415530dc06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:57.906747 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895b76f-d486-4524-bd47-69415530dc06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:57.906747 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895b76f-d486-4524-bd47-69415530dc06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:57.906747 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895b76f-d486-4524-bd47-69415530dc06 admin admin] PATH_INFO: `/v3/projects/637a2c210f25416e8492fbbb82929c3c/users/2af0fca4120c4e11a38f9213b9dddce7/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:57.934384 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 897/1774] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:04:57 2022] PUT /identity/v3/projects/05b45b43b383499da00bff9eabb1f5a8/users/5d830eeb1bc44e5bbd4e160ad7003725/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:57.935974 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b36fd-d70e-42fd-a6c4-1a1ad5750ea9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:57.936263 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b36fd-d70e-42fd-a6c4-1a1ad5750ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:57.936542 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b36fd-d70e-42fd-a6c4-1a1ad5750ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:57.937842 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2895b76f-d486-4524-bd47-69415530dc06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=637a2c210f25416e8492fbbb82929c3c, user_id=2af0fca4120c4e11a38f9213b9dddce7, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:57.941654 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2895b76f-d486-4524-bd47-69415530dc06 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:57.975110 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 878/1775] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:04:57 2022] PUT /identity/v3/projects/637a2c210f25416e8492fbbb82929c3c/users/2af0fca4120c4e11a38f9213b9dddce7/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:57.977667 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2895b76f-d486-4524-bd47-69415530dc06 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:57.979814 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2895b76f-d486-4524-bd47-69415530dc06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:58.000750 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d31b36fd-d70e-42fd-a6c4-1a1ad5750ea9 None None] MFA Rules not processed for user `5d830eeb1bc44e5bbd4e160ad7003725`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:58.006035 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d31b36fd-d70e-42fd-a6c4-1a1ad5750ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:58.096661 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:58.122950 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2895b76f-d486-4524-bd47-69415530dc06 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:58.136096 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28e2bad-d485-4ca9-b9a8-6bfa2d9089f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:58.136096 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e2bad-d485-4ca9-b9a8-6bfa2d9089f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:58.136096 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e2bad-d485-4ca9-b9a8-6bfa2d9089f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:58.136096 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e2bad-d485-4ca9-b9a8-6bfa2d9089f8 admin admin] PATH_INFO: `/v3/projects/cef8be2cb85e49b687bfd50b22a9a091` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:58.136454 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28e2bad-d485-4ca9-b9a8-6bfa2d9089f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cef8be2cb85e49b687bfd50b22a9a091)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:58.137902 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28e2bad-d485-4ca9-b9a8-6bfa2d9089f8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:58.161722 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 898/1776] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:57 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:58.162703 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b36fd-d70e-42fd-a6c4-1a1ad5750ea9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:58.176351 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b36fd-d70e-42fd-a6c4-1a1ad5750ea9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:58.182567 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a318f4d-f0d0-45a8-8690-189a323d0136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:58.183361 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a318f4d-f0d0-45a8-8690-189a323d0136 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:58.183892 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a318f4d-f0d0-45a8-8690-189a323d0136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:58.184305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a318f4d-f0d0-45a8-8690-189a323d0136 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:58.186929 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a318f4d-f0d0-45a8-8690-189a323d0136 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:58.189839 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a318f4d-f0d0-45a8-8690-189a323d0136 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:58.201714 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 899/1777] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:58.202606 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a318f4d-f0d0-45a8-8690-189a323d0136 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:58.206189 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3a318f4d-f0d0-45a8-8690-189a323d0136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:58.222880 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b28e2bad-d485-4ca9-b9a8-6bfa2d9089f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cef8be2cb85e49b687bfd50b22a9a091', 'request_id': 'req-b28e2bad-d485-4ca9-b9a8-6bfa2d9089f8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:58.231107 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b28e2bad-d485-4ca9-b9a8-6bfa2d9089f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cef8be2cb85e49b687bfd50b22a9a091', 'request_id': 'req-b28e2bad-d485-4ca9-b9a8-6bfa2d9089f8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:58.255310 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b28e2bad-d485-4ca9-b9a8-6bfa2d9089f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cef8be2cb85e49b687bfd50b22a9a091', 'request_id': 'req-b28e2bad-d485-4ca9-b9a8-6bfa2d9089f8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:58.266671 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b28e2bad-d485-4ca9-b9a8-6bfa2d9089f8 admin admin] The token cache is being invalidate because project cef8be2cb85e49b687bfd50b22a9a091 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:04:58.267160 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b28e2bad-d485-4ca9-b9a8-6bfa2d9089f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:58.267814 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b28e2bad-d485-4ca9-b9a8-6bfa2d9089f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:58.269495 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 879/1778] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:04:57 2022] DELETE /identity/v3/projects/cef8be2cb85e49b687bfd50b22a9a091 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:58.270521 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28e2bad-d485-4ca9-b9a8-6bfa2d9089f8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:58.272059 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b28e2bad-d485-4ca9-b9a8-6bfa2d9089f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:58.277625 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:58.291411 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a318f4d-f0d0-45a8-8690-189a323d0136 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:58.340886 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28e2bad-d485-4ca9-b9a8-6bfa2d9089f8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:58.341085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fcbdbf-34ab-4eac-9eb9-3bd94e02cf2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:58.341693 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fcbdbf-34ab-4eac-9eb9-3bd94e02cf2c service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:58.342111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fcbdbf-34ab-4eac-9eb9-3bd94e02cf2c service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:58.342407 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fcbdbf-34ab-4eac-9eb9-3bd94e02cf2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:58.344156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-96fcbdbf-34ab-4eac-9eb9-3bd94e02cf2c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:58.345766 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67deaf9d-3f07-4400-9315-eda54a1e9080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:58.346655 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67deaf9d-3f07-4400-9315-eda54a1e9080 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:58.346943 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67deaf9d-3f07-4400-9315-eda54a1e9080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:58.347224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67deaf9d-3f07-4400-9315-eda54a1e9080 admin admin] PATH_INFO: `/v3/projects/637a2c210f25416e8492fbbb82929c3c/users/2af0fca4120c4e11a38f9213b9dddce7/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:58.349474 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67deaf9d-3f07-4400-9315-eda54a1e9080 admin admin] RBAC: Authorizing `identity:create_grant(project_id=637a2c210f25416e8492fbbb82929c3c, user_id=2af0fca4120c4e11a38f9213b9dddce7, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:58.352748 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67deaf9d-3f07-4400-9315-eda54a1e9080 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:58.363049 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 880/1779] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:04:58 2022] PUT /identity/v3/projects/637a2c210f25416e8492fbbb82929c3c/users/2af0fca4120c4e11a38f9213b9dddce7/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:58.363854 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67deaf9d-3f07-4400-9315-eda54a1e9080 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:58.365423 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-67deaf9d-3f07-4400-9315-eda54a1e9080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:58.423596 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fcbdbf-34ab-4eac-9eb9-3bd94e02cf2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:58.425061 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fcbdbf-34ab-4eac-9eb9-3bd94e02cf2c service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:58.437194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 900/1780] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:58 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:04:58.438784 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9fed4f-d32c-4a37-a9a9-d46739708ae8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:58.439076 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9fed4f-d32c-4a37-a9a9-d46739708ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:58.439358 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9fed4f-d32c-4a37-a9a9-d46739708ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:58.459777 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2b9fed4f-d32c-4a37-a9a9-d46739708ae8 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:58.462344 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2b9fed4f-d32c-4a37-a9a9-d46739708ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:58.466762 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67deaf9d-3f07-4400-9315-eda54a1e9080 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:58.471977 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa3905-fe23-4ac9-b57f-434742e464f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:58.472862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa3905-fe23-4ac9-b57f-434742e464f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:58.473539 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa3905-fe23-4ac9-b57f-434742e464f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:58.474025 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa3905-fe23-4ac9-b57f-434742e464f6 admin admin] PATH_INFO: `/v3/users/1e61ec5021c84fdc849ee222f764e76b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:58.475755 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 901/1781] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:58.476515 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa3905-fe23-4ac9-b57f-434742e464f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e61ec5021c84fdc849ee222f764e76b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:58.480240 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e3dc0-0a58-4dd5-8d55-baf2a1fdcfca None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:58.480509 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa3905-fe23-4ac9-b57f-434742e464f6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:58.480617 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e3dc0-0a58-4dd5-8d55-baf2a1fdcfca None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:58.480834 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e3dc0-0a58-4dd5-8d55-baf2a1fdcfca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:58.510572 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-857e3dc0-0a58-4dd5-8d55-baf2a1fdcfca None None] MFA Rules not processed for user `2af0fca4120c4e11a38f9213b9dddce7`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:58.511907 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-857e3dc0-0a58-4dd5-8d55-baf2a1fdcfca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:58.525305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:58.541350 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-59fa3905-fe23-4ac9-b57f-434742e464f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e61ec5021c84fdc849ee222f764e76b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:58.542139 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-59fa3905-fe23-4ac9-b57f-434742e464f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e61ec5021c84fdc849ee222f764e76b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:58.542933 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-59fa3905-fe23-4ac9-b57f-434742e464f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e61ec5021c84fdc849ee222f764e76b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:58.550752 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-59fa3905-fe23-4ac9-b57f-434742e464f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e61ec5021c84fdc849ee222f764e76b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:58.558110 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-59fa3905-fe23-4ac9-b57f-434742e464f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e61ec5021c84fdc849ee222f764e76b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:58.576993 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 881/1782] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:04:58 2022] DELETE /identity/v3/users/1e61ec5021c84fdc849ee222f764e76b => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:58.578385 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579f148-57fa-42f3-9261-2043d363f5b5 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:58.578673 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579f148-57fa-42f3-9261-2043d363f5b5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:58.578942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579f148-57fa-42f3-9261-2043d363f5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:58.583120 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 902/1783] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:58 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:58.583965 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e3dc0-0a58-4dd5-8d55-baf2a1fdcfca None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:58.585255 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-857e3dc0-0a58-4dd5-8d55-baf2a1fdcfca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:58.599989 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1579f148-57fa-42f3-9261-2043d363f5b5 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:58.601457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1579f148-57fa-42f3-9261-2043d363f5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:58.611372 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:58.645659 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e3dc0-0a58-4dd5-8d55-baf2a1fdcfca None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:58.664414 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 882/1784] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:58.665737 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82826227-c732-4f45-be34-ca659a4fcc35 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:58.666033 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82826227-c732-4f45-be34-ca659a4fcc35 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:58.666316 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82826227-c732-4f45-be34-ca659a4fcc35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:58.685751 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-82826227-c732-4f45-be34-ca659a4fcc35 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:58.687033 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-82826227-c732-4f45-be34-ca659a4fcc35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:58.696638 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 883/1785] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:58.697518 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82826227-c732-4f45-be34-ca659a4fcc35 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:58.699174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-82826227-c732-4f45-be34-ca659a4fcc35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:58.704144 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d05a99-c725-4eb1-a685-5e3ac7d5adc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:58.705082 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d05a99-c725-4eb1-a685-5e3ac7d5adc0 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:58.705473 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d05a99-c725-4eb1-a685-5e3ac7d5adc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:58.705903 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d05a99-c725-4eb1-a685-5e3ac7d5adc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:58.708980 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-99d05a99-c725-4eb1-a685-5e3ac7d5adc0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:58.713013 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82826227-c732-4f45-be34-ca659a4fcc35 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:58.716606 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb04c7-574f-4480-a552-4c0a422c2b31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:58.717261 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb04c7-574f-4480-a552-4c0a422c2b31 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:58.717547 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb04c7-574f-4480-a552-4c0a422c2b31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:58.717887 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb04c7-574f-4480-a552-4c0a422c2b31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:58.774207 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d05a99-c725-4eb1-a685-5e3ac7d5adc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:58.775664 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d05a99-c725-4eb1-a685-5e3ac7d5adc0 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:58.784711 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb04c7-574f-4480-a552-4c0a422c2b31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:58.784883 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 903/1786] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:58 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:04:58.785983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d05a99-c725-4eb1-a685-5e3ac7d5adc0 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:58.786583 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb04c7-574f-4480-a552-4c0a422c2b31 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:58.797025 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 884/1787] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:58 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:04:58.798706 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb04c7-574f-4480-a552-4c0a422c2b31 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:58.798856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d05a99-c725-4eb1-a685-5e3ac7d5adc0 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:58.800049 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0fbb04c7-574f-4480-a552-4c0a422c2b31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:58.803633 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7675e-82f4-4cb9-b139-35579542710c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:58.804549 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7675e-82f4-4cb9-b139-35579542710c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:58.804748 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7675e-82f4-4cb9-b139-35579542710c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:58.805144 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7675e-82f4-4cb9-b139-35579542710c admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:58.806228 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7675e-82f4-4cb9-b139-35579542710c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:04:58.806544 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7675e-82f4-4cb9-b139-35579542710c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:58.808154 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7675e-82f4-4cb9-b139-35579542710c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:58.824352 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 904/1788] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:04:58.829053 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7675e-82f4-4cb9-b139-35579542710c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:58.856901 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7675e-82f4-4cb9-b139-35579542710c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:58.863492 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e29d03-bcdf-4c5d-8f64-a378b27d3123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:58.864168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e29d03-bcdf-4c5d-8f64-a378b27d3123 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:58.864923 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e29d03-bcdf-4c5d-8f64-a378b27d3123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:58.865202 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e29d03-bcdf-4c5d-8f64-a378b27d3123 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:58.866190 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e29d03-bcdf-4c5d-8f64-a378b27d3123 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:58.868665 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e29d03-bcdf-4c5d-8f64-a378b27d3123 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:58.873162 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:58.886070 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb04c7-574f-4480-a552-4c0a422c2b31 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:58.897031 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ce10c-0fc7-4bad-9bcf-e7dc514bf99b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:58.897541 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ce10c-0fc7-4bad-9bcf-e7dc514bf99b service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:58.897857 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ce10c-0fc7-4bad-9bcf-e7dc514bf99b service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:58.898144 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ce10c-0fc7-4bad-9bcf-e7dc514bf99b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:58.908757 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 905/1789] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:04:58 2022] POST /identity/v3/projects => generated 369 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:04:58.913122 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e29d03-bcdf-4c5d-8f64-a378b27d3123 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:58.921227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313ce10c-0fc7-4bad-9bcf-e7dc514bf99b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:58.922885 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313ce10c-0fc7-4bad-9bcf-e7dc514bf99b service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:58.941618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 885/1790] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:04:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:04:59.012355 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e29d03-bcdf-4c5d-8f64-a378b27d3123 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:59.023967 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce03c4b0-4da0-47e1-82d7-37d8b577abc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:59.025763 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03c4b0-4da0-47e1-82d7-37d8b577abc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:59.026268 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03c4b0-4da0-47e1-82d7-37d8b577abc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:59.026694 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03c4b0-4da0-47e1-82d7-37d8b577abc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:59.027820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce03c4b0-4da0-47e1-82d7-37d8b577abc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:59.030254 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce03c4b0-4da0-47e1-82d7-37d8b577abc4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:59.077100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 906/1791] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:04:58 2022] POST /identity/v3/users => generated 285 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:59.081961 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ce10c-0fc7-4bad-9bcf-e7dc514bf99b service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:59.102790 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ce10c-0fc7-4bad-9bcf-e7dc514bf99b service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:59.119466 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df7eeb-554f-4cfe-b72d-1926c25b3d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:59.120199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df7eeb-554f-4cfe-b72d-1926c25b3d85 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:59.120534 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df7eeb-554f-4cfe-b72d-1926c25b3d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:59.120843 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df7eeb-554f-4cfe-b72d-1926c25b3d85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:59.121607 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df7eeb-554f-4cfe-b72d-1926c25b3d85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:59.123337 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df7eeb-554f-4cfe-b72d-1926c25b3d85 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:59.150386 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 886/1792] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:59.153916 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce03c4b0-4da0-47e1-82d7-37d8b577abc4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:59.170970 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce03c4b0-4da0-47e1-82d7-37d8b577abc4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:59.175816 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd55e00-f8b6-4b16-b8e5-7cf53bfd5686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:59.176665 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd55e00-f8b6-4b16-b8e5-7cf53bfd5686 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:59.178511 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd55e00-f8b6-4b16-b8e5-7cf53bfd5686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:59.178904 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd55e00-f8b6-4b16-b8e5-7cf53bfd5686 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:59.179986 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd55e00-f8b6-4b16-b8e5-7cf53bfd5686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:59.184280 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd55e00-f8b6-4b16-b8e5-7cf53bfd5686 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:59.202242 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 907/1793] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:59.211895 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df7eeb-554f-4cfe-b72d-1926c25b3d85 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:59.226074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df7eeb-554f-4cfe-b72d-1926c25b3d85 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:59.232414 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ca7b3-c237-421d-b0ae-bfe1be0f7466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:59.233792 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ca7b3-c237-421d-b0ae-bfe1be0f7466 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:59.234345 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ca7b3-c237-421d-b0ae-bfe1be0f7466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:59.234889 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ca7b3-c237-421d-b0ae-bfe1be0f7466 admin admin] PATH_INFO: `/v3/projects/5ba022c31e33473082273f1e1ea6e0f1/users/f3ecce19e3c34dd09d1af94afa9e72d6/roles/54195e660fc349978b39d5794765a095` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:59.253251 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277ca7b3-c237-421d-b0ae-bfe1be0f7466 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ba022c31e33473082273f1e1ea6e0f1, user_id=f3ecce19e3c34dd09d1af94afa9e72d6, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:59.257085 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277ca7b3-c237-421d-b0ae-bfe1be0f7466 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:59.303214 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 887/1794] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:04:59 2022] PUT /identity/v3/projects/5ba022c31e33473082273f1e1ea6e0f1/users/f3ecce19e3c34dd09d1af94afa9e72d6/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:59.313540 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0eb4f-0148-46a2-b1a8-4158a2dc2db9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:59.313887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0eb4f-0148-46a2-b1a8-4158a2dc2db9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:59.314142 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0eb4f-0148-46a2-b1a8-4158a2dc2db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:59.375205 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-eff0eb4f-0148-46a2-b1a8-4158a2dc2db9 None None] MFA Rules not processed for user `f3ecce19e3c34dd09d1af94afa9e72d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:59.378009 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-eff0eb4f-0148-46a2-b1a8-4158a2dc2db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:59.460324 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ca7b3-c237-421d-b0ae-bfe1be0f7466 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:59.463088 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-277ca7b3-c237-421d-b0ae-bfe1be0f7466 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:59.510328 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:59.514610 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 908/1795] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:59 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:59.515573 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff0eb4f-0148-46a2-b1a8-4158a2dc2db9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:59.606001 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:04:59.624206 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff0eb4f-0148-46a2-b1a8-4158a2dc2db9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:59.626474 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ca7b3-c237-421d-b0ae-bfe1be0f7466 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:59.635266 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5e3df-cb99-4f6c-a9b6-850dab193589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:59.636419 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5e3df-cb99-4f6c-a9b6-850dab193589 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:59.636639 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5e3df-cb99-4f6c-a9b6-850dab193589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:59.636951 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5e3df-cb99-4f6c-a9b6-850dab193589 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:59.640716 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a5e3df-cb99-4f6c-a9b6-850dab193589 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:59.640828 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59213ac-ff6d-4a91-a102-a9cb94250d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:59.640828 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59213ac-ff6d-4a91-a102-a9cb94250d35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:59.640828 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59213ac-ff6d-4a91-a102-a9cb94250d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:59.641062 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a5e3df-cb99-4f6c-a9b6-850dab193589 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:59.641144 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59213ac-ff6d-4a91-a102-a9cb94250d35 admin admin] PATH_INFO: `/v3/projects/37aea02c7b974071a39754881deb779a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:59.641463 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59213ac-ff6d-4a91-a102-a9cb94250d35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37aea02c7b974071a39754881deb779a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:59.646033 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59213ac-ff6d-4a91-a102-a9cb94250d35 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:59.673201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 909/1796] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:04:59 2022] POST /identity/v3/projects => generated 367 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:59.677901 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5e3df-cb99-4f6c-a9b6-850dab193589 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:59.702172 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e59213ac-ff6d-4a91-a102-a9cb94250d35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37aea02c7b974071a39754881deb779a', 'request_id': 'req-e59213ac-ff6d-4a91-a102-a9cb94250d35'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:59.708918 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e59213ac-ff6d-4a91-a102-a9cb94250d35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37aea02c7b974071a39754881deb779a', 'request_id': 'req-e59213ac-ff6d-4a91-a102-a9cb94250d35'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:59.722031 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e59213ac-ff6d-4a91-a102-a9cb94250d35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37aea02c7b974071a39754881deb779a', 'request_id': 'req-e59213ac-ff6d-4a91-a102-a9cb94250d35'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:59.735397 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e59213ac-ff6d-4a91-a102-a9cb94250d35 admin admin] The token cache is being invalidate because project 37aea02c7b974071a39754881deb779a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:04:59.736004 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e59213ac-ff6d-4a91-a102-a9cb94250d35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:59.736515 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e59213ac-ff6d-4a91-a102-a9cb94250d35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:04:59.738183 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 888/1797] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:04:59 2022] DELETE /identity/v3/projects/37aea02c7b974071a39754881deb779a => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:04:59.743040 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f61542-05d4-468f-8031-6768011d2847 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:59.743411 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f61542-05d4-468f-8031-6768011d2847 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:59.743750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f61542-05d4-468f-8031-6768011d2847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:59.751701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5e3df-cb99-4f6c-a9b6-850dab193589 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:59.772482 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-46f61542-05d4-468f-8031-6768011d2847 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:59.774351 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-46f61542-05d4-468f-8031-6768011d2847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:59.838941 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfce22a-4c65-4d2c-840e-3882e001d96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:59.839446 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfce22a-4c65-4d2c-840e-3882e001d96e admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:59.839767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfce22a-4c65-4d2c-840e-3882e001d96e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:59.841838 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfce22a-4c65-4d2c-840e-3882e001d96e admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:59.842718 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfce22a-4c65-4d2c-840e-3882e001d96e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:59.845010 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfce22a-4c65-4d2c-840e-3882e001d96e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:59.847690 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 889/1798] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:04:59.856411 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a122b46-4d2b-46f7-a3c1-9522e080a092 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:59.856706 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a122b46-4d2b-46f7-a3c1-9522e080a092 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:59.856994 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a122b46-4d2b-46f7-a3c1-9522e080a092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:59.867828 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 910/1799] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:04:59 2022] POST /identity/v3/users => generated 290 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:04:59.872535 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfce22a-4c65-4d2c-840e-3882e001d96e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:59.875219 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bbfce22a-4c65-4d2c-840e-3882e001d96e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:59.880392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1a122b46-4d2b-46f7-a3c1-9522e080a092 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:04:59.881918 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1a122b46-4d2b-46f7-a3c1-9522e080a092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:04:59.889758 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfce22a-4c65-4d2c-840e-3882e001d96e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:59.893066 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 890/1800] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:04:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:04:59.896972 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7804e-c265-43fd-9246-f80c3033503d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:59.897337 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a122b46-4d2b-46f7-a3c1-9522e080a092 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:59.897670 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7804e-c265-43fd-9246-f80c3033503d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:59.897967 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7804e-c265-43fd-9246-f80c3033503d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:59.898993 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7804e-c265-43fd-9246-f80c3033503d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:59.899663 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b7804e-c265-43fd-9246-f80c3033503d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:59.901278 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b7804e-c265-43fd-9246-f80c3033503d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:59.911487 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a122b46-4d2b-46f7-a3c1-9522e080a092 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:59.917442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 911/1801] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:04:59.917961 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317efae-afe8-428b-958b-e635c816bf85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:59.918617 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317efae-afe8-428b-958b-e635c816bf85 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:59.918907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317efae-afe8-428b-958b-e635c816bf85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:59.919184 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317efae-afe8-428b-958b-e635c816bf85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:59.919925 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2317efae-afe8-428b-958b-e635c816bf85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:04:59.920234 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2317efae-afe8-428b-958b-e635c816bf85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:59.926454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7804e-c265-43fd-9246-f80c3033503d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:59.926554 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2317efae-afe8-428b-958b-e635c816bf85 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:59.932775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7804e-c265-43fd-9246-f80c3033503d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:59.937829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f6015c-a731-4f75-909a-c17313a88466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:59.938409 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 891/1802] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 16:04:59.938716 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f6015c-a731-4f75-909a-c17313a88466 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:59.939003 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f6015c-a731-4f75-909a-c17313a88466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:59.939294 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f6015c-a731-4f75-909a-c17313a88466 admin admin] PATH_INFO: `/v3/projects/a4de7cf8aa104c5aa135f60467a5ba10/users/31548b3ce8f54284803098ca8cd4c36f/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:59.942525 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317efae-afe8-428b-958b-e635c816bf85 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:59.953512 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317efae-afe8-428b-958b-e635c816bf85 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:04:59.958372 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67310db7-628e-46c6-92d0-9361b65218dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:04:59.959030 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67310db7-628e-46c6-92d0-9361b65218dd admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:04:59.959216 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f6015c-a731-4f75-909a-c17313a88466 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4de7cf8aa104c5aa135f60467a5ba10, user_id=31548b3ce8f54284803098ca8cd4c36f, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:59.959339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67310db7-628e-46c6-92d0-9361b65218dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:04:59.959597 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67310db7-628e-46c6-92d0-9361b65218dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:04:59.960308 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67310db7-628e-46c6-92d0-9361b65218dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:04:59.963014 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f6015c-a731-4f75-909a-c17313a88466 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:59.963131 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67310db7-628e-46c6-92d0-9361b65218dd admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:04:59.980701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 912/1803] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:04:59 2022] PUT /identity/v3/projects/a4de7cf8aa104c5aa135f60467a5ba10/users/31548b3ce8f54284803098ca8cd4c36f/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:04:59.986707 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f6015c-a731-4f75-909a-c17313a88466 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:04:59.993630 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:00.015946 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 892/1804] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:04:59 2022] POST /identity/v3/projects => generated 385 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:05:00.015946 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67310db7-628e-46c6-92d0-9361b65218dd admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:00.049353 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:00.074645 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f6015c-a731-4f75-909a-c17313a88466 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:00.109339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67310db7-628e-46c6-92d0-9361b65218dd admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:00.116189 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b7bf5-4656-40e9-9290-090decf0c461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:00.116921 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b7bf5-4656-40e9-9290-090decf0c461 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:00.117180 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b7bf5-4656-40e9-9290-090decf0c461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:00.117520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b7bf5-4656-40e9-9290-090decf0c461 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:00.118294 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b7bf5-4656-40e9-9290-090decf0c461 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:00.120742 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b7bf5-4656-40e9-9290-090decf0c461 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:00.145708 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 893/1805] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:00 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:00.147778 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b7bf5-4656-40e9-9290-090decf0c461 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:00.153793 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4fdfd7-32f0-42cc-a06b-f1d80af8f6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:00.154546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4fdfd7-32f0-42cc-a06b-f1d80af8f6b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:00.154928 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4fdfd7-32f0-42cc-a06b-f1d80af8f6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:00.155242 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4fdfd7-32f0-42cc-a06b-f1d80af8f6b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:00.156095 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4fdfd7-32f0-42cc-a06b-f1d80af8f6b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:00.157975 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4fdfd7-32f0-42cc-a06b-f1d80af8f6b8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:00.162082 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b7bf5-4656-40e9-9290-090decf0c461 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:00.167942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa7e8ef-837c-4882-82b5-fc50d296d3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:00.168745 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa7e8ef-837c-4882-82b5-fc50d296d3d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:00.169034 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa7e8ef-837c-4882-82b5-fc50d296d3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:00.169613 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa7e8ef-837c-4882-82b5-fc50d296d3d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:00.170362 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa7e8ef-837c-4882-82b5-fc50d296d3d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:00.172010 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa7e8ef-837c-4882-82b5-fc50d296d3d2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:00.173663 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 913/1806] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:00.177929 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4fdfd7-32f0-42cc-a06b-f1d80af8f6b8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:00.185325 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 894/1807] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 16:05:00.188089 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4fdfd7-32f0-42cc-a06b-f1d80af8f6b8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:00.189849 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa7e8ef-837c-4882-82b5-fc50d296d3d2 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:00.193068 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9463328-df64-463e-8940-df30933a6900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:00.194083 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9463328-df64-463e-8940-df30933a6900 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:00.194369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9463328-df64-463e-8940-df30933a6900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:00.194671 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9463328-df64-463e-8940-df30933a6900 admin admin] PATH_INFO: `/v3/projects/a4de7cf8aa104c5aa135f60467a5ba10/users/31548b3ce8f54284803098ca8cd4c36f/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:00.197256 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9463328-df64-463e-8940-df30933a6900 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4de7cf8aa104c5aa135f60467a5ba10, user_id=31548b3ce8f54284803098ca8cd4c36f, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:00.200296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9463328-df64-463e-8940-df30933a6900 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:00.201317 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa7e8ef-837c-4882-82b5-fc50d296d3d2 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:00.206276 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e12b1c-4a48-4c41-a8d6-283688fc76a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:00.206938 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e12b1c-4a48-4c41-a8d6-283688fc76a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:00.207225 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e12b1c-4a48-4c41-a8d6-283688fc76a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:00.207504 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e12b1c-4a48-4c41-a8d6-283688fc76a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:00.208229 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e12b1c-4a48-4c41-a8d6-283688fc76a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:00.209934 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e12b1c-4a48-4c41-a8d6-283688fc76a1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:00.211955 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 914/1808] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:00 2022] PUT /identity/v3/projects/a4de7cf8aa104c5aa135f60467a5ba10/users/31548b3ce8f54284803098ca8cd4c36f/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:00.223313 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb7d70-b879-4c26-a51a-83e0ebf8e053 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:00.223610 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb7d70-b879-4c26-a51a-83e0ebf8e053 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:00.223897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb7d70-b879-4c26-a51a-83e0ebf8e053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:00.227242 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 895/1809] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 16:05:00.230879 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e12b1c-4a48-4c41-a8d6-283688fc76a1 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:00.241238 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e12b1c-4a48-4c41-a8d6-283688fc76a1 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:00.256719 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-51fb7d70-b879-4c26-a51a-83e0ebf8e053 None None] MFA Rules not processed for user `31548b3ce8f54284803098ca8cd4c36f`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:00.258439 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-51fb7d70-b879-4c26-a51a-83e0ebf8e053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:00.314550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76b4c2-d23d-4900-8e07-4cce4d4f286b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:00.315463 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76b4c2-d23d-4900-8e07-4cce4d4f286b admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:00.315755 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76b4c2-d23d-4900-8e07-4cce4d4f286b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:00.316042 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76b4c2-d23d-4900-8e07-4cce4d4f286b admin admin] PATH_INFO: `/v3/projects/c601011de1264a779bf1f3396ad966cf/users/62fbfe5944864289b6655b511a6eb921/roles/54195e660fc349978b39d5794765a095` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:00.334128 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b76b4c2-d23d-4900-8e07-4cce4d4f286b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c601011de1264a779bf1f3396ad966cf, user_id=62fbfe5944864289b6655b511a6eb921, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:00.337552 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b76b4c2-d23d-4900-8e07-4cce4d4f286b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:00.337964 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 915/1810] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:00 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:05:00.351558 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb7d70-b879-4c26-a51a-83e0ebf8e053 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:00.353045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-51fb7d70-b879-4c26-a51a-83e0ebf8e053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:00.364442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 896/1811] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:00 2022] PUT /identity/v3/projects/c601011de1264a779bf1f3396ad966cf/users/62fbfe5944864289b6655b511a6eb921/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:05:00.375631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962febe-dcda-4e6a-a492-b09957c08513 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:00.376054 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962febe-dcda-4e6a-a492-b09957c08513 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:00.376341 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962febe-dcda-4e6a-a492-b09957c08513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:00.407400 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f962febe-dcda-4e6a-a492-b09957c08513 None None] MFA Rules not processed for user `62fbfe5944864289b6655b511a6eb921`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:00.409066 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f962febe-dcda-4e6a-a492-b09957c08513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:00.433730 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb7d70-b879-4c26-a51a-83e0ebf8e053 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:00.442630 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:00.456262 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:00.490080 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 897/1812] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:05:00.495468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962febe-dcda-4e6a-a492-b09957c08513 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:00.498196 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813c012-f7dc-4550-9c4a-6bddc4a713f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:00.498903 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813c012-f7dc-4550-9c4a-6bddc4a713f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:00.499246 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813c012-f7dc-4550-9c4a-6bddc4a713f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:00.501431 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813c012-f7dc-4550-9c4a-6bddc4a713f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:00.596215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962febe-dcda-4e6a-a492-b09957c08513 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:00.602963 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb82013-e801-4366-aa0b-ac1864de2d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:00.604376 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb82013-e801-4366-aa0b-ac1864de2d29 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:00.604743 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb82013-e801-4366-aa0b-ac1864de2d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:00.605271 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb82013-e801-4366-aa0b-ac1864de2d29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:00.606309 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb82013-e801-4366-aa0b-ac1864de2d29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:00.607519 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2813c012-f7dc-4550-9c4a-6bddc4a713f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:00.609198 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2813c012-f7dc-4550-9c4a-6bddc4a713f6 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:00.609324 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb82013-e801-4366-aa0b-ac1864de2d29 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:00.638921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 916/1813] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:00 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:00.669920 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 898/1814] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:00 2022] POST /identity/v3/projects => generated 385 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:05:00.671077 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813c012-f7dc-4550-9c4a-6bddc4a713f6 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:00.686171 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813c012-f7dc-4550-9c4a-6bddc4a713f6 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:00.694284 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb82013-e801-4366-aa0b-ac1864de2d29 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:00.763556 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae4434-f6b2-4b0f-acd6-3a4d43b98033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:00.764136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae4434-f6b2-4b0f-acd6-3a4d43b98033 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:00.765013 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae4434-f6b2-4b0f-acd6-3a4d43b98033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:00.765013 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae4434-f6b2-4b0f-acd6-3a4d43b98033 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:00.765436 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae4434-f6b2-4b0f-acd6-3a4d43b98033 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:00.767403 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae4434-f6b2-4b0f-acd6-3a4d43b98033 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:00.790788 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 917/1815] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:00 2022] POST /identity/v3/users => generated 299 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:00.794419 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae4434-f6b2-4b0f-acd6-3a4d43b98033 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:00.806430 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb82013-e801-4366-aa0b-ac1864de2d29 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:00.818370 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae4434-f6b2-4b0f-acd6-3a4d43b98033 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:00.821452 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa2eda-f3f3-48f5-b9db-ea85eea72ba3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:00.822151 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa2eda-f3f3-48f5-b9db-ea85eea72ba3 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:00.822439 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa2eda-f3f3-48f5-b9db-ea85eea72ba3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:00.822719 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa2eda-f3f3-48f5-b9db-ea85eea72ba3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:00.825665 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-94fa2eda-f3f3-48f5-b9db-ea85eea72ba3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:00.834543 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995fe36-8a29-4278-9f64-d8a4d6cf4b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:00.835236 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995fe36-8a29-4278-9f64-d8a4d6cf4b90 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:00.835507 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995fe36-8a29-4278-9f64-d8a4d6cf4b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:00.836340 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995fe36-8a29-4278-9f64-d8a4d6cf4b90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:00.840713 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0995fe36-8a29-4278-9f64-d8a4d6cf4b90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:00.842395 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0995fe36-8a29-4278-9f64-d8a4d6cf4b90 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:00.853604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 918/1816] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 16:05:00.854565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995fe36-8a29-4278-9f64-d8a4d6cf4b90 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:00.856250 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0995fe36-8a29-4278-9f64-d8a4d6cf4b90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:00.869537 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995fe36-8a29-4278-9f64-d8a4d6cf4b90 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:00.875547 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de6a06-2fe5-4d37-95e6-8919cbc8f571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:00.884039 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de6a06-2fe5-4d37-95e6-8919cbc8f571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:00.884427 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de6a06-2fe5-4d37-95e6-8919cbc8f571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:00.885692 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de6a06-2fe5-4d37-95e6-8919cbc8f571 admin admin] PATH_INFO: `/v3/users/d67f09592176445fbe84218772502f17` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:00.887451 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5de6a06-2fe5-4d37-95e6-8919cbc8f571 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d67f09592176445fbe84218772502f17)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:00.889699 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5de6a06-2fe5-4d37-95e6-8919cbc8f571 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:00.932844 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa2eda-f3f3-48f5-b9db-ea85eea72ba3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:00.934711 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa2eda-f3f3-48f5-b9db-ea85eea72ba3 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:00.946546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:00.948625 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 899/1817] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:00 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 16:05:00.949412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa2eda-f3f3-48f5-b9db-ea85eea72ba3 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:00.963568 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e5de6a06-2fe5-4d37-95e6-8919cbc8f571 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd67f09592176445fbe84218772502f17'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:00.964709 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e5de6a06-2fe5-4d37-95e6-8919cbc8f571 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd67f09592176445fbe84218772502f17'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:00.965596 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e5de6a06-2fe5-4d37-95e6-8919cbc8f571 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd67f09592176445fbe84218772502f17'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:00.966434 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa2eda-f3f3-48f5-b9db-ea85eea72ba3 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:00.975011 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e5de6a06-2fe5-4d37-95e6-8919cbc8f571 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd67f09592176445fbe84218772502f17'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:00.979645 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321a7cf-c5e4-4163-b73f-2e3e842538e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:00.980567 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321a7cf-c5e4-4163-b73f-2e3e842538e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:00.980860 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321a7cf-c5e4-4163-b73f-2e3e842538e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:00.981177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321a7cf-c5e4-4163-b73f-2e3e842538e1 admin admin] PATH_INFO: `/v3/projects/e3cf534e70a54d68975982c045ed9463/users/7756e2d7172c488da720bd23f5843680/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:00.987791 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e5de6a06-2fe5-4d37-95e6-8919cbc8f571 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd67f09592176445fbe84218772502f17'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:01.002929 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2321a7cf-c5e4-4163-b73f-2e3e842538e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3cf534e70a54d68975982c045ed9463, user_id=7756e2d7172c488da720bd23f5843680, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:01.006280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2321a7cf-c5e4-4163-b73f-2e3e842538e1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:01.010969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:01.013192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 919/1818] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:00 2022] DELETE /identity/v3/users/d67f09592176445fbe84218772502f17 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:01.018142 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01ea79-c263-4982-9b7a-44b3dd8e6959 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:01.018769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01ea79-c263-4982-9b7a-44b3dd8e6959 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:01.019531 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01ea79-c263-4982-9b7a-44b3dd8e6959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:01.043490 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 900/1819] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:00 2022] PUT /identity/v3/projects/e3cf534e70a54d68975982c045ed9463/users/7756e2d7172c488da720bd23f5843680/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:01.049387 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3d01ea79-c263-4982-9b7a-44b3dd8e6959 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:01.049520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321a7cf-c5e4-4163-b73f-2e3e842538e1 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:01.050699 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3d01ea79-c263-4982-9b7a-44b3dd8e6959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:01.052148 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2321a7cf-c5e4-4163-b73f-2e3e842538e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:01.127975 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 920/1820] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:01.140041 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01ea79-c263-4982-9b7a-44b3dd8e6959 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:01.142171 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3d01ea79-c263-4982-9b7a-44b3dd8e6959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:01.148966 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321a7cf-c5e4-4163-b73f-2e3e842538e1 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:01.154724 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67908e6c-0c85-42dc-ba01-6468c3c755bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:01.155570 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67908e6c-0c85-42dc-ba01-6468c3c755bf admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:01.155889 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67908e6c-0c85-42dc-ba01-6468c3c755bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:01.156328 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67908e6c-0c85-42dc-ba01-6468c3c755bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:01.157072 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67908e6c-0c85-42dc-ba01-6468c3c755bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:01.158947 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67908e6c-0c85-42dc-ba01-6468c3c755bf admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:01.174622 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 901/1821] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:01.179312 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67908e6c-0c85-42dc-ba01-6468c3c755bf admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:01.189190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67908e6c-0c85-42dc-ba01-6468c3c755bf admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:01.196078 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40807aa-5cbb-465c-9bb1-6ebea5c6bd32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:01.196969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40807aa-5cbb-465c-9bb1-6ebea5c6bd32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:01.197255 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40807aa-5cbb-465c-9bb1-6ebea5c6bd32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:01.197937 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40807aa-5cbb-465c-9bb1-6ebea5c6bd32 admin admin] PATH_INFO: `/v3/projects/e3cf534e70a54d68975982c045ed9463/users/7756e2d7172c488da720bd23f5843680/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:01.200827 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40807aa-5cbb-465c-9bb1-6ebea5c6bd32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3cf534e70a54d68975982c045ed9463, user_id=7756e2d7172c488da720bd23f5843680, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:01.203861 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40807aa-5cbb-465c-9bb1-6ebea5c6bd32 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:01.215359 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01ea79-c263-4982-9b7a-44b3dd8e6959 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:01.217644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 902/1822] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:01 2022] PUT /identity/v3/projects/e3cf534e70a54d68975982c045ed9463/users/7756e2d7172c488da720bd23f5843680/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:05:01.229834 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f6341-a212-4bfe-b606-c84f5f7e2e7b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:01.230129 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f6341-a212-4bfe-b606-c84f5f7e2e7b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:01.230678 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f6341-a212-4bfe-b606-c84f5f7e2e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:01.263130 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a62f6341-a212-4bfe-b606-c84f5f7e2e7b None None] MFA Rules not processed for user `7756e2d7172c488da720bd23f5843680`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:01.264268 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a62f6341-a212-4bfe-b606-c84f5f7e2e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:01.282633 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff90cd-bd2a-4361-a5c5-d2e5af3b31fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:01.282633 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff90cd-bd2a-4361-a5c5-d2e5af3b31fc service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:01.282633 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff90cd-bd2a-4361-a5c5-d2e5af3b31fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:01.282633 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff90cd-bd2a-4361-a5c5-d2e5af3b31fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:01.308761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:01.364295 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff90cd-bd2a-4361-a5c5-d2e5af3b31fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:01.365820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff90cd-bd2a-4361-a5c5-d2e5af3b31fc service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:01.370300 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 903/1823] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:01.380160 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 921/1824] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:01.382270 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f6341-a212-4bfe-b606-c84f5f7e2e7b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:01.383799 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a62f6341-a212-4bfe-b606-c84f5f7e2e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:01.454816 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f6341-a212-4bfe-b606-c84f5f7e2e7b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:01.459595 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e3ca2-2e3a-4a47-93e1-51d60c103cf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:01.460467 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e3ca2-2e3a-4a47-93e1-51d60c103cf1 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:01.461445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e3ca2-2e3a-4a47-93e1-51d60c103cf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:01.461787 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e3ca2-2e3a-4a47-93e1-51d60c103cf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:01.472146 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314e3ca2-2e3a-4a47-93e1-51d60c103cf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:01.473905 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314e3ca2-2e3a-4a47-93e1-51d60c103cf1 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:01.485495 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 904/1825] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:01 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 16:05:01.515262 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff90cd-bd2a-4361-a5c5-d2e5af3b31fc service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:01.525889 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff90cd-bd2a-4361-a5c5-d2e5af3b31fc service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:01.532108 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b6d03-f248-4608-8211-596a94f461b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:01.533184 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b6d03-f248-4608-8211-596a94f461b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:01.533476 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b6d03-f248-4608-8211-596a94f461b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:01.534204 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b6d03-f248-4608-8211-596a94f461b2 admin admin] PATH_INFO: `/v3/users/62fbfe5944864289b6655b511a6eb921` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:01.537964 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b6d03-f248-4608-8211-596a94f461b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62fbfe5944864289b6655b511a6eb921)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:01.540588 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b6d03-f248-4608-8211-596a94f461b2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:01.608735 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-078b6d03-f248-4608-8211-596a94f461b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62fbfe5944864289b6655b511a6eb921'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:01.609681 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-078b6d03-f248-4608-8211-596a94f461b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62fbfe5944864289b6655b511a6eb921'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:01.610285 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-078b6d03-f248-4608-8211-596a94f461b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62fbfe5944864289b6655b511a6eb921'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:01.622918 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-078b6d03-f248-4608-8211-596a94f461b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62fbfe5944864289b6655b511a6eb921'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:01.631990 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-078b6d03-f248-4608-8211-596a94f461b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62fbfe5944864289b6655b511a6eb921'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:01.653754 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 922/1826] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:01 2022] DELETE /identity/v3/users/62fbfe5944864289b6655b511a6eb921 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:01.658647 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326a42d-7099-4d8b-af69-a66ff96704dd None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:01.659070 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326a42d-7099-4d8b-af69-a66ff96704dd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:01.659486 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326a42d-7099-4d8b-af69-a66ff96704dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:01.679811 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:01.684896 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b6d03-f248-4608-8211-596a94f461b2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:01.687172 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-078b6d03-f248-4608-8211-596a94f461b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:01.691057 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f326a42d-7099-4d8b-af69-a66ff96704dd None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:01.693099 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f326a42d-7099-4d8b-af69-a66ff96704dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:01.775465 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b6d03-f248-4608-8211-596a94f461b2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:01.780649 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a205b-6067-4612-9440-830358cb3c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:01.781555 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a205b-6067-4612-9440-830358cb3c8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:01.781871 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a205b-6067-4612-9440-830358cb3c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:01.782158 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 905/1827] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:01.782256 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a205b-6067-4612-9440-830358cb3c8d admin admin] PATH_INFO: `/v3/projects/9fffbf9dacce4a6c8e7e807474c39cd7` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:01.783618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a205b-6067-4612-9440-830358cb3c8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fffbf9dacce4a6c8e7e807474c39cd7)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:01.785759 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a205b-6067-4612-9440-830358cb3c8d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:01.792956 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326a42d-7099-4d8b-af69-a66ff96704dd None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:01.794442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f326a42d-7099-4d8b-af69-a66ff96704dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:01.824293 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:01.837803 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1b8a205b-6067-4612-9440-830358cb3c8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fffbf9dacce4a6c8e7e807474c39cd7', 'request_id': 'req-1b8a205b-6067-4612-9440-830358cb3c8d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:01.845626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1b8a205b-6067-4612-9440-830358cb3c8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fffbf9dacce4a6c8e7e807474c39cd7', 'request_id': 'req-1b8a205b-6067-4612-9440-830358cb3c8d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:01.852232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1b8a205b-6067-4612-9440-830358cb3c8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fffbf9dacce4a6c8e7e807474c39cd7', 'request_id': 'req-1b8a205b-6067-4612-9440-830358cb3c8d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:01.858116 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326a42d-7099-4d8b-af69-a66ff96704dd None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:01.867042 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1b8a205b-6067-4612-9440-830358cb3c8d admin admin] The token cache is being invalidate because project 9fffbf9dacce4a6c8e7e807474c39cd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:01.867336 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1b8a205b-6067-4612-9440-830358cb3c8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:01.868331 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1b8a205b-6067-4612-9440-830358cb3c8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:01.869799 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 923/1828] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:01 2022] DELETE /identity/v3/projects/9fffbf9dacce4a6c8e7e807474c39cd7 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:01.874053 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a205b-6067-4612-9440-830358cb3c8d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:01.875916 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1b8a205b-6067-4612-9440-830358cb3c8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:01.915970 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636adc1e-99e1-41fd-a587-db9f0eaddfb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:01.916625 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636adc1e-99e1-41fd-a587-db9f0eaddfb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:01.916911 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636adc1e-99e1-41fd-a587-db9f0eaddfb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:01.917187 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636adc1e-99e1-41fd-a587-db9f0eaddfb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:01.919032 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-636adc1e-99e1-41fd-a587-db9f0eaddfb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:01.955883 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a205b-6067-4612-9440-830358cb3c8d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:01.961731 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb14a9-6102-47a2-94da-51778aa55176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:01.962713 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb14a9-6102-47a2-94da-51778aa55176 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:01.963065 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb14a9-6102-47a2-94da-51778aa55176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:01.963478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb14a9-6102-47a2-94da-51778aa55176 admin admin] PATH_INFO: `/v3/users/2af0fca4120c4e11a38f9213b9dddce7` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:01.965023 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb14a9-6102-47a2-94da-51778aa55176 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2af0fca4120c4e11a38f9213b9dddce7)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:01.967033 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb14a9-6102-47a2-94da-51778aa55176 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:01.988939 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636adc1e-99e1-41fd-a587-db9f0eaddfb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:01.990658 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636adc1e-99e1-41fd-a587-db9f0eaddfb9 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:02.004722 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 906/1829] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 16:05:02.038629 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3dbb14a9-6102-47a2-94da-51778aa55176 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2af0fca4120c4e11a38f9213b9dddce7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:02.039722 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3dbb14a9-6102-47a2-94da-51778aa55176 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2af0fca4120c4e11a38f9213b9dddce7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:02.040562 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3dbb14a9-6102-47a2-94da-51778aa55176 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2af0fca4120c4e11a38f9213b9dddce7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:02.049105 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3dbb14a9-6102-47a2-94da-51778aa55176 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2af0fca4120c4e11a38f9213b9dddce7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:02.061020 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3dbb14a9-6102-47a2-94da-51778aa55176 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2af0fca4120c4e11a38f9213b9dddce7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:02.096423 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 924/1830] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:01 2022] DELETE /identity/v3/users/2af0fca4120c4e11a38f9213b9dddce7 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:02.454388 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636adc1e-99e1-41fd-a587-db9f0eaddfb9 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:02.466428 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636adc1e-99e1-41fd-a587-db9f0eaddfb9 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:02.497914 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb14a9-6102-47a2-94da-51778aa55176 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:02.499919 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3dbb14a9-6102-47a2-94da-51778aa55176 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:02.511994 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:02.566303 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:02.586273 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09c82b-6e38-4e1a-9c67-07bd2a8b109a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:02.587146 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09c82b-6e38-4e1a-9c67-07bd2a8b109a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:02.587440 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09c82b-6e38-4e1a-9c67-07bd2a8b109a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:02.587726 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09c82b-6e38-4e1a-9c67-07bd2a8b109a admin admin] PATH_INFO: `/v3/projects/637a2c210f25416e8492fbbb82929c3c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:02.588950 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb09c82b-6e38-4e1a-9c67-07bd2a8b109a admin admin] RBAC: Authorizing `identity:delete_project(project_id=637a2c210f25416e8492fbbb82929c3c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:02.591002 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb09c82b-6e38-4e1a-9c67-07bd2a8b109a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:02.595120 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb14a9-6102-47a2-94da-51778aa55176 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:02.602581 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca3aa1-1564-40b7-a391-4a0e9b504641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:02.604121 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca3aa1-1564-40b7-a391-4a0e9b504641 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:02.604413 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca3aa1-1564-40b7-a391-4a0e9b504641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:02.604693 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca3aa1-1564-40b7-a391-4a0e9b504641 admin admin] PATH_INFO: `/v3/projects/c601011de1264a779bf1f3396ad966cf` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:02.605951 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca3aa1-1564-40b7-a391-4a0e9b504641 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c601011de1264a779bf1f3396ad966cf)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:02.607939 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca3aa1-1564-40b7-a391-4a0e9b504641 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:02.645406 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cb09c82b-6e38-4e1a-9c67-07bd2a8b109a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '637a2c210f25416e8492fbbb82929c3c', 'request_id': 'req-cb09c82b-6e38-4e1a-9c67-07bd2a8b109a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:02.653269 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cb09c82b-6e38-4e1a-9c67-07bd2a8b109a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '637a2c210f25416e8492fbbb82929c3c', 'request_id': 'req-cb09c82b-6e38-4e1a-9c67-07bd2a8b109a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:02.661989 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cb09c82b-6e38-4e1a-9c67-07bd2a8b109a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '637a2c210f25416e8492fbbb82929c3c', 'request_id': 'req-cb09c82b-6e38-4e1a-9c67-07bd2a8b109a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:02.667118 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-acca3aa1-1564-40b7-a391-4a0e9b504641 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c601011de1264a779bf1f3396ad966cf', 'request_id': 'req-acca3aa1-1564-40b7-a391-4a0e9b504641'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:02.676197 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-acca3aa1-1564-40b7-a391-4a0e9b504641 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c601011de1264a779bf1f3396ad966cf', 'request_id': 'req-acca3aa1-1564-40b7-a391-4a0e9b504641'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:02.677602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cb09c82b-6e38-4e1a-9c67-07bd2a8b109a admin admin] The token cache is being invalidate because project 637a2c210f25416e8492fbbb82929c3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:02.677928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cb09c82b-6e38-4e1a-9c67-07bd2a8b109a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:02.678588 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cb09c82b-6e38-4e1a-9c67-07bd2a8b109a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:02.680512 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 907/1831] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:02 2022] DELETE /identity/v3/projects/637a2c210f25416e8492fbbb82929c3c => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:02.683256 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-acca3aa1-1564-40b7-a391-4a0e9b504641 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c601011de1264a779bf1f3396ad966cf', 'request_id': 'req-acca3aa1-1564-40b7-a391-4a0e9b504641'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:02.685021 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d60f5-ac3e-4165-8ee5-dbf8438967b0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:02.685314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d60f5-ac3e-4165-8ee5-dbf8438967b0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:02.685629 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d60f5-ac3e-4165-8ee5-dbf8438967b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:02.697818 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-acca3aa1-1564-40b7-a391-4a0e9b504641 admin admin] The token cache is being invalidate because project c601011de1264a779bf1f3396ad966cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:02.698113 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-acca3aa1-1564-40b7-a391-4a0e9b504641 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:02.699253 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-acca3aa1-1564-40b7-a391-4a0e9b504641 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:02.701176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 925/1832] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:02 2022] DELETE /identity/v3/projects/c601011de1264a779bf1f3396ad966cf => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:02.704728 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-bf0d60f5-ac3e-4165-8ee5-dbf8438967b0 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:02.705252 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca3aa1-1564-40b7-a391-4a0e9b504641 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:02.705995 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bf0d60f5-ac3e-4165-8ee5-dbf8438967b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:02.706820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-acca3aa1-1564-40b7-a391-4a0e9b504641 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:02.787698 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca3aa1-1564-40b7-a391-4a0e9b504641 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:02.790362 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 908/1833] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:05:02.794094 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17679c0a-8764-42a1-9dcb-0077e42da390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:02.794983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17679c0a-8764-42a1-9dcb-0077e42da390 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:02.795264 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17679c0a-8764-42a1-9dcb-0077e42da390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:02.795541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17679c0a-8764-42a1-9dcb-0077e42da390 admin admin] PATH_INFO: `/v3/users/7756e2d7172c488da720bd23f5843680` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:02.797181 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17679c0a-8764-42a1-9dcb-0077e42da390 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7756e2d7172c488da720bd23f5843680)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:02.800201 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d21b9d-589a-4dcd-9392-3039de561ab6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:02.800331 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17679c0a-8764-42a1-9dcb-0077e42da390 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:02.800539 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d21b9d-589a-4dcd-9392-3039de561ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:02.800823 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d21b9d-589a-4dcd-9392-3039de561ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:02.825052 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-37d21b9d-589a-4dcd-9392-3039de561ab6 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:02.826468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-37d21b9d-589a-4dcd-9392-3039de561ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:02.849660 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 909/1834] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:02.853959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d21b9d-589a-4dcd-9392-3039de561ab6 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:02.869273 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d21b9d-589a-4dcd-9392-3039de561ab6 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:02.874536 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7ad2b-1f94-4fd5-b8de-5913d5df9805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:02.875206 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7ad2b-1f94-4fd5-b8de-5913d5df9805 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:02.875646 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7ad2b-1f94-4fd5-b8de-5913d5df9805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:02.875962 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7ad2b-1f94-4fd5-b8de-5913d5df9805 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:02.876704 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a7ad2b-1f94-4fd5-b8de-5913d5df9805 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:05:02.877182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a7ad2b-1f94-4fd5-b8de-5913d5df9805 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:02.878300 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-17679c0a-8764-42a1-9dcb-0077e42da390 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7756e2d7172c488da720bd23f5843680'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:02.879243 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-17679c0a-8764-42a1-9dcb-0077e42da390 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7756e2d7172c488da720bd23f5843680'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:02.879968 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-17679c0a-8764-42a1-9dcb-0077e42da390 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7756e2d7172c488da720bd23f5843680'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:02.880279 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a7ad2b-1f94-4fd5-b8de-5913d5df9805 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:02.890410 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-17679c0a-8764-42a1-9dcb-0077e42da390 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7756e2d7172c488da720bd23f5843680'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:02.899719 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-17679c0a-8764-42a1-9dcb-0077e42da390 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7756e2d7172c488da720bd23f5843680'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:02.900130 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 910/1835] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:05:02.918110 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7ad2b-1f94-4fd5-b8de-5913d5df9805 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:02.919840 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c7a7ad2b-1f94-4fd5-b8de-5913d5df9805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:02.928600 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:02.944234 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7ad2b-1f94-4fd5-b8de-5913d5df9805 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:02.949830 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 926/1836] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:02 2022] DELETE /identity/v3/users/7756e2d7172c488da720bd23f5843680 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:02.966280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d925a-1a83-4761-a8dc-3ae64288a130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:02.966993 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d925a-1a83-4761-a8dc-3ae64288a130 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:02.967284 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d925a-1a83-4761-a8dc-3ae64288a130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:02.967565 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d925a-1a83-4761-a8dc-3ae64288a130 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:02.968282 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96d925a-1a83-4761-a8dc-3ae64288a130 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:02.970228 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96d925a-1a83-4761-a8dc-3ae64288a130 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:03.010820 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 911/1837] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:02 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:03.012236 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17679c0a-8764-42a1-9dcb-0077e42da390 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:03.094857 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17679c0a-8764-42a1-9dcb-0077e42da390 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:03.175448 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fceef38-aedf-4c31-8eb8-bcb80c1807b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:03.176197 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fceef38-aedf-4c31-8eb8-bcb80c1807b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:03.176597 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fceef38-aedf-4c31-8eb8-bcb80c1807b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:03.176920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fceef38-aedf-4c31-8eb8-bcb80c1807b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:03.177764 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fceef38-aedf-4c31-8eb8-bcb80c1807b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:03.180553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fceef38-aedf-4c31-8eb8-bcb80c1807b3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:03.217269 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 927/1838] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:03 2022] POST /identity/v3/users => generated 290 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:03.221178 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d925a-1a83-4761-a8dc-3ae64288a130 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:03.232213 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:03.254397 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d925a-1a83-4761-a8dc-3ae64288a130 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:03.268667 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa783b27-2c2c-4f18-91c2-8b4919f6793a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:03.269905 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa783b27-2c2c-4f18-91c2-8b4919f6793a admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:03.270197 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa783b27-2c2c-4f18-91c2-8b4919f6793a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:03.270481 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa783b27-2c2c-4f18-91c2-8b4919f6793a admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:03.271456 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa783b27-2c2c-4f18-91c2-8b4919f6793a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:03.274704 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa783b27-2c2c-4f18-91c2-8b4919f6793a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:03.295296 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 912/1839] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:03.299263 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fceef38-aedf-4c31-8eb8-bcb80c1807b3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:03.318342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fceef38-aedf-4c31-8eb8-bcb80c1807b3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:03.325632 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de8480-1338-4f9e-8d93-c59f334c8c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:03.327183 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de8480-1338-4f9e-8d93-c59f334c8c55 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:03.328157 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de8480-1338-4f9e-8d93-c59f334c8c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:03.328635 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de8480-1338-4f9e-8d93-c59f334c8c55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:03.329346 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa783b27-2c2c-4f18-91c2-8b4919f6793a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:03.329964 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40de8480-1338-4f9e-8d93-c59f334c8c55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:03.331295 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fa783b27-2c2c-4f18-91c2-8b4919f6793a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:03.332800 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40de8480-1338-4f9e-8d93-c59f334c8c55 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:03.349370 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa783b27-2c2c-4f18-91c2-8b4919f6793a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:03.354117 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a8d6f-1481-498e-afca-4bbcee525914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:03.355134 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a8d6f-1481-498e-afca-4bbcee525914 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:03.355427 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a8d6f-1481-498e-afca-4bbcee525914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:03.355710 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a8d6f-1481-498e-afca-4bbcee525914 admin admin] PATH_INFO: `/v3/projects/e3cf534e70a54d68975982c045ed9463` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:03.357465 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0a8d6f-1481-498e-afca-4bbcee525914 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3cf534e70a54d68975982c045ed9463)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:03.358672 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 928/1840] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 16:05:03.360114 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0a8d6f-1481-498e-afca-4bbcee525914 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:03.363596 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de8480-1338-4f9e-8d93-c59f334c8c55 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:03.382320 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de8480-1338-4f9e-8d93-c59f334c8c55 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:03.388432 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a1bd4-0460-4a49-ad2e-24822cfbe94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:03.389505 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a1bd4-0460-4a49-ad2e-24822cfbe94d admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:03.389883 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a1bd4-0460-4a49-ad2e-24822cfbe94d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:03.390162 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a1bd4-0460-4a49-ad2e-24822cfbe94d admin admin] PATH_INFO: `/v3/projects/706be92d077f412bbf64e7f0e674ad6b/users/ccedd4a8370e4c51a0be88be92458237/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:03.412728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a1bd4-0460-4a49-ad2e-24822cfbe94d admin admin] RBAC: Authorizing `identity:create_grant(project_id=706be92d077f412bbf64e7f0e674ad6b, user_id=ccedd4a8370e4c51a0be88be92458237, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:03.417876 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a1bd4-0460-4a49-ad2e-24822cfbe94d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:03.442360 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 929/1841] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:03 2022] PUT /identity/v3/projects/706be92d077f412bbf64e7f0e674ad6b/users/ccedd4a8370e4c51a0be88be92458237/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:03.458187 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02c33e-7c1a-496c-9389-0d4dec23848a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:03.458470 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02c33e-7c1a-496c-9389-0d4dec23848a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:03.458756 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02c33e-7c1a-496c-9389-0d4dec23848a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:03.467892 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4a0a8d6f-1481-498e-afca-4bbcee525914 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3cf534e70a54d68975982c045ed9463', 'request_id': 'req-4a0a8d6f-1481-498e-afca-4bbcee525914'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:03.476249 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4a0a8d6f-1481-498e-afca-4bbcee525914 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3cf534e70a54d68975982c045ed9463', 'request_id': 'req-4a0a8d6f-1481-498e-afca-4bbcee525914'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:03.488313 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4a0a8d6f-1481-498e-afca-4bbcee525914 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3cf534e70a54d68975982c045ed9463', 'request_id': 'req-4a0a8d6f-1481-498e-afca-4bbcee525914'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:03.507104 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5e02c33e-7c1a-496c-9389-0d4dec23848a None None] MFA Rules not processed for user `ccedd4a8370e4c51a0be88be92458237`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:03.509848 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5e02c33e-7c1a-496c-9389-0d4dec23848a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:03.512008 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4a0a8d6f-1481-498e-afca-4bbcee525914 admin admin] The token cache is being invalidate because project e3cf534e70a54d68975982c045ed9463 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:03.512749 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4a0a8d6f-1481-498e-afca-4bbcee525914 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:03.513738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4a0a8d6f-1481-498e-afca-4bbcee525914 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:03.515983 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 913/1842] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:03 2022] DELETE /identity/v3/projects/e3cf534e70a54d68975982c045ed9463 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:05:03.523356 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d635e6-9923-498a-bd73-1ab2f7d48cea None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:03.523927 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d635e6-9923-498a-bd73-1ab2f7d48cea None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:03.524277 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d635e6-9923-498a-bd73-1ab2f7d48cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:03.547127 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a0d635e6-9923-498a-bd73-1ab2f7d48cea None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:03.548706 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a0d635e6-9923-498a-bd73-1ab2f7d48cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:03.583750 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:03.640695 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 914/1843] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:03.643538 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 930/1844] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:05:03.647753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d635e6-9923-498a-bd73-1ab2f7d48cea None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:03.650360 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e73865-67b9-4f8d-aa70-b0810beb4396 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:03.650730 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a0d635e6-9923-498a-bd73-1ab2f7d48cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:03.650845 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e73865-67b9-4f8d-aa70-b0810beb4396 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:03.651055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e73865-67b9-4f8d-aa70-b0810beb4396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:03.663002 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d635e6-9923-498a-bd73-1ab2f7d48cea None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:03.668529 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1942a-c973-4443-a682-75db24401802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:03.669330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1942a-c973-4443-a682-75db24401802 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:03.669878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1942a-c973-4443-a682-75db24401802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:03.670247 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1942a-c973-4443-a682-75db24401802 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:03.671063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc1942a-c973-4443-a682-75db24401802 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:03.671196 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-78e73865-67b9-4f8d-aa70-b0810beb4396 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:03.672389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-78e73865-67b9-4f8d-aa70-b0810beb4396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:03.673187 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc1942a-c973-4443-a682-75db24401802 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:03.684068 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 931/1845] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:03.685374 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e73865-67b9-4f8d-aa70-b0810beb4396 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:03.697985 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e73865-67b9-4f8d-aa70-b0810beb4396 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:03.702708 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db41b9-d772-4e58-96c0-fcca10c6d43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:03.703366 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db41b9-d772-4e58-96c0-fcca10c6d43d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:03.703817 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db41b9-d772-4e58-96c0-fcca10c6d43d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:03.704154 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db41b9-d772-4e58-96c0-fcca10c6d43d admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:03.704259 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 915/1846] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:03 2022] POST /identity/v3/projects => generated 379 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:03.704883 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26db41b9-d772-4e58-96c0-fcca10c6d43d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:05:03.705193 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26db41b9-d772-4e58-96c0-fcca10c6d43d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:03.706847 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26db41b9-d772-4e58-96c0-fcca10c6d43d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:03.707719 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1942a-c973-4443-a682-75db24401802 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:03.719751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 932/1847] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:05:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 16:05:03.724067 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db41b9-d772-4e58-96c0-fcca10c6d43d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:03.734889 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db41b9-d772-4e58-96c0-fcca10c6d43d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:03.783070 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:03.799271 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1942a-c973-4443-a682-75db24401802 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:03.801762 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623750d0-0e13-4a58-a9b4-799fd2189577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:03.802352 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623750d0-0e13-4a58-a9b4-799fd2189577 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:03.802682 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623750d0-0e13-4a58-a9b4-799fd2189577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:03.803028 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623750d0-0e13-4a58-a9b4-799fd2189577 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:03.804077 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623750d0-0e13-4a58-a9b4-799fd2189577 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:03.806444 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623750d0-0e13-4a58-a9b4-799fd2189577 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:03.811653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1926f63-6494-4cd4-8f3f-fedbd2de6c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:03.812509 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1926f63-6494-4cd4-8f3f-fedbd2de6c40 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:03.812809 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1926f63-6494-4cd4-8f3f-fedbd2de6c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:03.813473 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1926f63-6494-4cd4-8f3f-fedbd2de6c40 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:03.814356 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1926f63-6494-4cd4-8f3f-fedbd2de6c40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:03.816362 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1926f63-6494-4cd4-8f3f-fedbd2de6c40 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:03.841312 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 916/1848] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:03 2022] POST /identity/v3/users => generated 296 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:03.842330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1926f63-6494-4cd4-8f3f-fedbd2de6c40 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:03.846639 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 933/1849] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:03 2022] POST /identity/v3/projects => generated 379 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:05:03.847631 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623750d0-0e13-4a58-a9b4-799fd2189577 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:03.852904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1926f63-6494-4cd4-8f3f-fedbd2de6c40 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:03.921945 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623750d0-0e13-4a58-a9b4-799fd2189577 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:03.924165 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a21fc9-7b2e-438b-aab0-5cfabb28b052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:03.924808 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a21fc9-7b2e-438b-aab0-5cfabb28b052 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:03.925055 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a21fc9-7b2e-438b-aab0-5cfabb28b052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:03.925343 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a21fc9-7b2e-438b-aab0-5cfabb28b052 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:03.926842 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a21fc9-7b2e-438b-aab0-5cfabb28b052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:03.927987 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a21fc9-7b2e-438b-aab0-5cfabb28b052 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:03.928433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124c6ca-5120-4f97-9704-0ec198f2c59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:03.931058 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124c6ca-5120-4f97-9704-0ec198f2c59d admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:03.931058 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124c6ca-5120-4f97-9704-0ec198f2c59d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:03.931058 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124c6ca-5120-4f97-9704-0ec198f2c59d admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:03.931755 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f124c6ca-5120-4f97-9704-0ec198f2c59d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:03.934839 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f124c6ca-5120-4f97-9704-0ec198f2c59d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:03.953874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 917/1850] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:03.956263 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a21fc9-7b2e-438b-aab0-5cfabb28b052 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:03.966454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 934/1851] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:03 2022] POST /identity/v3/users => generated 290 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:05:03.970401 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a21fc9-7b2e-438b-aab0-5cfabb28b052 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:03.970732 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124c6ca-5120-4f97-9704-0ec198f2c59d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:03.976114 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f1ddb-b0e6-43a4-aa98-eaa9a6e18108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:03.977048 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f1ddb-b0e6-43a4-aa98-eaa9a6e18108 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:03.977358 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f1ddb-b0e6-43a4-aa98-eaa9a6e18108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:03.977738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f1ddb-b0e6-43a4-aa98-eaa9a6e18108 admin admin] PATH_INFO: `/v3/projects/b3c99cfd46b64ceeb40d464180fac8de/users/4af8704137834929a2b48e5d8a0659e6/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:03.989788 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124c6ca-5120-4f97-9704-0ec198f2c59d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:03.999111 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f1ddb-b0e6-43a4-aa98-eaa9a6e18108 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3c99cfd46b64ceeb40d464180fac8de, user_id=4af8704137834929a2b48e5d8a0659e6, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:04.002204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f1ddb-b0e6-43a4-aa98-eaa9a6e18108 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:04.003846 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11798ed7-ba21-4637-a199-d1dcf06e726f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:04.005837 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11798ed7-ba21-4637-a199-d1dcf06e726f admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:04.009227 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11798ed7-ba21-4637-a199-d1dcf06e726f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:04.009862 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11798ed7-ba21-4637-a199-d1dcf06e726f admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:04.011479 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11798ed7-ba21-4637-a199-d1dcf06e726f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:04.016979 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11798ed7-ba21-4637-a199-d1dcf06e726f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:04.027540 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 918/1852] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:03 2022] PUT /identity/v3/projects/b3c99cfd46b64ceeb40d464180fac8de/users/4af8704137834929a2b48e5d8a0659e6/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:04.029950 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f1ddb-b0e6-43a4-aa98-eaa9a6e18108 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:04.043179 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 935/1853] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 16:05:04.047172 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11798ed7-ba21-4637-a199-d1dcf06e726f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:04.057788 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11798ed7-ba21-4637-a199-d1dcf06e726f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:04.081650 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:04.121132 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f1ddb-b0e6-43a4-aa98-eaa9a6e18108 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:04.124412 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febb994-279a-4306-a19e-49ad7ed0ed7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:04.125088 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febb994-279a-4306-a19e-49ad7ed0ed7b admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:04.125547 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febb994-279a-4306-a19e-49ad7ed0ed7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:04.125899 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febb994-279a-4306-a19e-49ad7ed0ed7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:04.126262 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730790c-3a6f-47ad-8171-3a2477784d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:04.126636 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6febb994-279a-4306-a19e-49ad7ed0ed7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:04.127146 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730790c-3a6f-47ad-8171-3a2477784d5d admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:04.127487 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730790c-3a6f-47ad-8171-3a2477784d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:04.127925 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730790c-3a6f-47ad-8171-3a2477784d5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:04.128347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6febb994-279a-4306-a19e-49ad7ed0ed7b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:04.128634 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f730790c-3a6f-47ad-8171-3a2477784d5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:04.130271 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f730790c-3a6f-47ad-8171-3a2477784d5d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:04.139997 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 936/1854] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 16:05:04.142978 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 919/1855] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:04.144580 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febb994-279a-4306-a19e-49ad7ed0ed7b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:04.147698 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730790c-3a6f-47ad-8171-3a2477784d5d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:04.154716 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febb994-279a-4306-a19e-49ad7ed0ed7b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:04.161537 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a090a137-f451-4d4c-a00b-b4777638a8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:04.162740 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090a137-f451-4d4c-a00b-b4777638a8f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:04.162740 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090a137-f451-4d4c-a00b-b4777638a8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:04.163020 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090a137-f451-4d4c-a00b-b4777638a8f2 admin admin] PATH_INFO: `/v3/projects/c03c670b45414ce09aa629295338f815/users/86fd92f728f74c79a1a5b54d2aa8bf86/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:04.166096 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730790c-3a6f-47ad-8171-3a2477784d5d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:04.181934 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9a5433-29aa-465c-a0a6-6eb5dfb8fffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:04.183863 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a090a137-f451-4d4c-a00b-b4777638a8f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c03c670b45414ce09aa629295338f815, user_id=86fd92f728f74c79a1a5b54d2aa8bf86, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:04.184464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a5433-29aa-465c-a0a6-6eb5dfb8fffb admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:04.185024 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a5433-29aa-465c-a0a6-6eb5dfb8fffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:04.187060 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a090a137-f451-4d4c-a00b-b4777638a8f2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:04.188088 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a5433-29aa-465c-a0a6-6eb5dfb8fffb admin admin] PATH_INFO: `/v3/projects/b3c99cfd46b64ceeb40d464180fac8de/users/4af8704137834929a2b48e5d8a0659e6/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:04.191257 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9a5433-29aa-465c-a0a6-6eb5dfb8fffb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3c99cfd46b64ceeb40d464180fac8de, user_id=4af8704137834929a2b48e5d8a0659e6, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:04.195520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9a5433-29aa-465c-a0a6-6eb5dfb8fffb admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:04.211400 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 920/1856] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:04 2022] PUT /identity/v3/projects/b3c99cfd46b64ceeb40d464180fac8de/users/4af8704137834929a2b48e5d8a0659e6/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:05:04.215782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 937/1857] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:04 2022] PUT /identity/v3/projects/c03c670b45414ce09aa629295338f815/users/86fd92f728f74c79a1a5b54d2aa8bf86/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:04.228485 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f9d12-6ca5-4185-8931-e7e88db7bf4e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:04.228615 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46865bb-8a4a-4a8d-9bb3-1665b3930b57 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:04.228721 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f9d12-6ca5-4185-8931-e7e88db7bf4e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:04.228902 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f9d12-6ca5-4185-8931-e7e88db7bf4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:04.229010 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46865bb-8a4a-4a8d-9bb3-1665b3930b57 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:04.229200 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46865bb-8a4a-4a8d-9bb3-1665b3930b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:04.273260 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c46865bb-8a4a-4a8d-9bb3-1665b3930b57 None None] MFA Rules not processed for user `4af8704137834929a2b48e5d8a0659e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:04.275143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c46865bb-8a4a-4a8d-9bb3-1665b3930b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:04.276303 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ac1f9d12-6ca5-4185-8931-e7e88db7bf4e None None] MFA Rules not processed for user `86fd92f728f74c79a1a5b54d2aa8bf86`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:04.277626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ac1f9d12-6ca5-4185-8931-e7e88db7bf4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:04.320602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:04.360905 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 938/1858] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:04.364904 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f9d12-6ca5-4185-8931-e7e88db7bf4e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:04.395069 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 921/1859] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:04 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:04.426626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46865bb-8a4a-4a8d-9bb3-1665b3930b57 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:04.428230 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c46865bb-8a4a-4a8d-9bb3-1665b3930b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:04.503873 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f9d12-6ca5-4185-8931-e7e88db7bf4e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:04.511733 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e8365a-a330-4705-80f2-d4422f41ce98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:04.512535 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e8365a-a330-4705-80f2-d4422f41ce98 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:04.512921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e8365a-a330-4705-80f2-d4422f41ce98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:04.513273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e8365a-a330-4705-80f2-d4422f41ce98 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:04.514095 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e8365a-a330-4705-80f2-d4422f41ce98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:04.520474 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e8365a-a330-4705-80f2-d4422f41ce98 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:04.579499 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 939/1860] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:04 2022] POST /identity/v3/projects => generated 377 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:05:04.583840 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e8365a-a330-4705-80f2-d4422f41ce98 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:04.608344 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46865bb-8a4a-4a8d-9bb3-1665b3930b57 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:04.627319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:04.721937 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2ca0ef-4cc8-465b-8136-eeb29b0e268d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:04.721937 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ca0ef-4cc8-465b-8136-eeb29b0e268d service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:04.721937 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ca0ef-4cc8-465b-8136-eeb29b0e268d service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:04.726875 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ca0ef-4cc8-465b-8136-eeb29b0e268d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:04.747853 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e8365a-a330-4705-80f2-d4422f41ce98 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:04.756157 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726eb152-d06f-4ad2-a48b-3480f47b7ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:04.757237 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726eb152-d06f-4ad2-a48b-3480f47b7ab7 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:04.757897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726eb152-d06f-4ad2-a48b-3480f47b7ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:04.758466 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726eb152-d06f-4ad2-a48b-3480f47b7ab7 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:04.759397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726eb152-d06f-4ad2-a48b-3480f47b7ab7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:04.762262 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726eb152-d06f-4ad2-a48b-3480f47b7ab7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:04.792128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 940/1861] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:04 2022] POST /identity/v3/users => generated 289 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:04.796501 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726eb152-d06f-4ad2-a48b-3480f47b7ab7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:04.821408 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726eb152-d06f-4ad2-a48b-3480f47b7ab7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:04.829465 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b938d-7712-4738-89b9-97369cb0edf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:04.830347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b938d-7712-4738-89b9-97369cb0edf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:04.830624 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b938d-7712-4738-89b9-97369cb0edf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:04.830906 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b938d-7712-4738-89b9-97369cb0edf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:04.832601 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8b938d-7712-4738-89b9-97369cb0edf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:04.834635 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8b938d-7712-4738-89b9-97369cb0edf6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:04.852305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 941/1862] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:04.856934 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b938d-7712-4738-89b9-97369cb0edf6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:04.869167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b938d-7712-4738-89b9-97369cb0edf6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:04.871681 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2ca0ef-4cc8-465b-8136-eeb29b0e268d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:04.873728 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2ca0ef-4cc8-465b-8136-eeb29b0e268d service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:04.874204 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c5e07-4dc7-40e2-8eb2-fdafe5adae29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:04.875099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c5e07-4dc7-40e2-8eb2-fdafe5adae29 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:04.875390 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c5e07-4dc7-40e2-8eb2-fdafe5adae29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:04.875817 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c5e07-4dc7-40e2-8eb2-fdafe5adae29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:04.876576 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308c5e07-4dc7-40e2-8eb2-fdafe5adae29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:04.878285 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308c5e07-4dc7-40e2-8eb2-fdafe5adae29 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:04.889628 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 922/1863] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:04 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 16:05:04.905136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 942/1864] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 16:05:04.907075 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2ca0ef-4cc8-465b-8136-eeb29b0e268d service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:04.921163 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2ca0ef-4cc8-465b-8136-eeb29b0e268d service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:04.927197 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1c3e2-7c82-4982-96c8-8cf6bf352705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:04.928197 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1c3e2-7c82-4982-96c8-8cf6bf352705 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:04.928799 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1c3e2-7c82-4982-96c8-8cf6bf352705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:04.928950 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1c3e2-7c82-4982-96c8-8cf6bf352705 admin admin] PATH_INFO: `/v3/projects/a6e9793ba4c44cebb349be26f7baba63/users/d094fb4900ca4da4ade96399402dee4b/roles/54195e660fc349978b39d5794765a095` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:04.947756 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c1c3e2-7c82-4982-96c8-8cf6bf352705 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6e9793ba4c44cebb349be26f7baba63, user_id=d094fb4900ca4da4ade96399402dee4b, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:04.950698 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c1c3e2-7c82-4982-96c8-8cf6bf352705 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:04.973609 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 923/1865] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:04 2022] PUT /identity/v3/projects/a6e9793ba4c44cebb349be26f7baba63/users/d094fb4900ca4da4ade96399402dee4b/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:04.987590 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ec9bb-def5-4d2a-9768-4c95786b64b3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:04.987880 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ec9bb-def5-4d2a-9768-4c95786b64b3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:04.988151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ec9bb-def5-4d2a-9768-4c95786b64b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:05.025104 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-780ec9bb-def5-4d2a-9768-4c95786b64b3 None None] MFA Rules not processed for user `d094fb4900ca4da4ade96399402dee4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:05.026958 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-780ec9bb-def5-4d2a-9768-4c95786b64b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:05.101526 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1c3e2-7c82-4982-96c8-8cf6bf352705 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:05.107183 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-45c1c3e2-7c82-4982-96c8-8cf6bf352705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:05.114892 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:05.149642 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 943/1866] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:05.155496 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ec9bb-def5-4d2a-9768-4c95786b64b3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:05.250729 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1c3e2-7c82-4982-96c8-8cf6bf352705 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:05.260433 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f1295-46c8-4085-aebb-a4e5f5328a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:05.261358 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f1295-46c8-4085-aebb-a4e5f5328a82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:05.261673 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f1295-46c8-4085-aebb-a4e5f5328a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:05.261957 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f1295-46c8-4085-aebb-a4e5f5328a82 admin admin] PATH_INFO: `/v3/users/f3ecce19e3c34dd09d1af94afa9e72d6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:05.263187 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f1295-46c8-4085-aebb-a4e5f5328a82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3ecce19e3c34dd09d1af94afa9e72d6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:05.265195 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f1295-46c8-4085-aebb-a4e5f5328a82 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:05.266790 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ec9bb-def5-4d2a-9768-4c95786b64b3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:05.279497 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707f6af-16b9-4c0b-9a62-e9b782f97913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:05.280625 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707f6af-16b9-4c0b-9a62-e9b782f97913 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:05.280922 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707f6af-16b9-4c0b-9a62-e9b782f97913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:05.281204 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707f6af-16b9-4c0b-9a62-e9b782f97913 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:05.282050 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707f6af-16b9-4c0b-9a62-e9b782f97913 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:05.286691 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707f6af-16b9-4c0b-9a62-e9b782f97913 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:05.307910 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:05.326172 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 944/1867] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:05 2022] POST /identity/v3/projects => generated 379 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:05.330384 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707f6af-16b9-4c0b-9a62-e9b782f97913 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:05.345411 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-489f1295-46c8-4085-aebb-a4e5f5328a82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3ecce19e3c34dd09d1af94afa9e72d6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:05.346240 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-489f1295-46c8-4085-aebb-a4e5f5328a82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3ecce19e3c34dd09d1af94afa9e72d6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:05.346882 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-489f1295-46c8-4085-aebb-a4e5f5328a82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3ecce19e3c34dd09d1af94afa9e72d6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:05.356502 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-489f1295-46c8-4085-aebb-a4e5f5328a82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3ecce19e3c34dd09d1af94afa9e72d6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:05.365994 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-489f1295-46c8-4085-aebb-a4e5f5328a82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3ecce19e3c34dd09d1af94afa9e72d6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:05.397693 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 924/1868] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:05 2022] DELETE /identity/v3/users/f3ecce19e3c34dd09d1af94afa9e72d6 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:05.407356 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc43f5-a700-41ee-88ea-cb3503df45fd None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:05.421038 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc43f5-a700-41ee-88ea-cb3503df45fd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:05.421490 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc43f5-a700-41ee-88ea-cb3503df45fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:05.447016 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707f6af-16b9-4c0b-9a62-e9b782f97913 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:05.478368 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-97fc43f5-a700-41ee-88ea-cb3503df45fd None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:05.480204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-97fc43f5-a700-41ee-88ea-cb3503df45fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:05.534509 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3068d66-f2b6-402f-a34e-b44af6d0c67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:05.536134 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3068d66-f2b6-402f-a34e-b44af6d0c67b admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:05.536892 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3068d66-f2b6-402f-a34e-b44af6d0c67b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:05.537421 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3068d66-f2b6-402f-a34e-b44af6d0c67b admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:05.541043 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3068d66-f2b6-402f-a34e-b44af6d0c67b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:05.543897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3068d66-f2b6-402f-a34e-b44af6d0c67b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:05.573291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 945/1869] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:05 2022] POST /identity/v3/users => generated 296 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:05:05.577622 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3068d66-f2b6-402f-a34e-b44af6d0c67b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:05.583147 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b3068d66-f2b6-402f-a34e-b44af6d0c67b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:05.607507 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 925/1870] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:05:05.608484 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc43f5-a700-41ee-88ea-cb3503df45fd None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:05.610234 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-97fc43f5-a700-41ee-88ea-cb3503df45fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:05.615565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3068d66-f2b6-402f-a34e-b44af6d0c67b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:05.626961 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0721b4-1186-4f5d-a8cd-007f5994e3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:05.627929 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0721b4-1186-4f5d-a8cd-007f5994e3d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:05.629084 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0721b4-1186-4f5d-a8cd-007f5994e3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:05.630481 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0721b4-1186-4f5d-a8cd-007f5994e3d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:05.632945 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0721b4-1186-4f5d-a8cd-007f5994e3d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:05.639721 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0721b4-1186-4f5d-a8cd-007f5994e3d6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:05.643417 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:05.661179 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 946/1871] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:05.661853 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0721b4-1186-4f5d-a8cd-007f5994e3d6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:05.667545 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0d0721b4-1186-4f5d-a8cd-007f5994e3d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:05.688610 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0721b4-1186-4f5d-a8cd-007f5994e3d6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:05.695214 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb166e07-7381-420c-b5de-df0b66beb048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:05.696196 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb166e07-7381-420c-b5de-df0b66beb048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:05.696403 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb166e07-7381-420c-b5de-df0b66beb048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:05.696683 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb166e07-7381-420c-b5de-df0b66beb048 admin admin] PATH_INFO: `/v3/users/eacadfd6d7bf4fb28a69df8315113d6f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:05.699849 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc43f5-a700-41ee-88ea-cb3503df45fd None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:05.704070 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb166e07-7381-420c-b5de-df0b66beb048 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eacadfd6d7bf4fb28a69df8315113d6f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:05.706100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb166e07-7381-420c-b5de-df0b66beb048 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:05.706618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43060e1d-8db2-4910-995b-5a510d4497e5 tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4af8704137834929a2b48e5d8a0659e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3c99cfd46b64ceeb40d464180fac8de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:05.707359 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43060e1d-8db2-4910-995b-5a510d4497e5 tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:05.707756 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43060e1d-8db2-4910-995b-5a510d4497e5 tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:05.708320 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43060e1d-8db2-4910-995b-5a510d4497e5 tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:05.709508 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43060e1d-8db2-4910-995b-5a510d4497e5 tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:05:05.709961 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43060e1d-8db2-4910-995b-5a510d4497e5 tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:05.712823 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43060e1d-8db2-4910-995b-5a510d4497e5 tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:05.734093 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 926/1872] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:05:05.735018 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43060e1d-8db2-4910-995b-5a510d4497e5 tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:05.753262 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43060e1d-8db2-4910-995b-5a510d4497e5 tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:05.759491 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319193c5-d44c-4ed8-8bb2-3180b65c120f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:05.760479 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319193c5-d44c-4ed8-8bb2-3180b65c120f admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:05.760767 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319193c5-d44c-4ed8-8bb2-3180b65c120f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:05.761099 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319193c5-d44c-4ed8-8bb2-3180b65c120f admin admin] PATH_INFO: `/v3/projects/8ee4156be29e4784bc71a50642807dbb/users/69f3459ed7344c2dbb465127dd662271/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:05.769921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eb166e07-7381-420c-b5de-df0b66beb048 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eacadfd6d7bf4fb28a69df8315113d6f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:05.770842 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eb166e07-7381-420c-b5de-df0b66beb048 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eacadfd6d7bf4fb28a69df8315113d6f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:05.771682 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eb166e07-7381-420c-b5de-df0b66beb048 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eacadfd6d7bf4fb28a69df8315113d6f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:05.783625 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319193c5-d44c-4ed8-8bb2-3180b65c120f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ee4156be29e4784bc71a50642807dbb, user_id=69f3459ed7344c2dbb465127dd662271, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:05.787510 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319193c5-d44c-4ed8-8bb2-3180b65c120f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:05.789809 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eb166e07-7381-420c-b5de-df0b66beb048 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eacadfd6d7bf4fb28a69df8315113d6f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:05.803106 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eb166e07-7381-420c-b5de-df0b66beb048 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eacadfd6d7bf4fb28a69df8315113d6f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:05.811576 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 927/1873] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:05 2022] PUT /identity/v3/projects/8ee4156be29e4784bc71a50642807dbb/users/69f3459ed7344c2dbb465127dd662271/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:05.814446 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319193c5-d44c-4ed8-8bb2-3180b65c120f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:05.814558 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:INSERT=5,keystone:SELECT=86,keystone:DELETE=9 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:05.816373 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-319193c5-d44c-4ed8-8bb2-3180b65c120f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:05.833369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 947/1874] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:05 2022] DELETE /identity/v3/users/eacadfd6d7bf4fb28a69df8315113d6f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:05.834827 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb166e07-7381-420c-b5de-df0b66beb048 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:05.837130 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-eb166e07-7381-420c-b5de-df0b66beb048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:05.893736 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319193c5-d44c-4ed8-8bb2-3180b65c120f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:05.925289 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb166e07-7381-420c-b5de-df0b66beb048 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:05.932747 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8404a1-ae61-4b0d-82a0-ad523cd15402 tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4af8704137834929a2b48e5d8a0659e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3c99cfd46b64ceeb40d464180fac8de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:05.933433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8404a1-ae61-4b0d-82a0-ad523cd15402 tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:05.933752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8404a1-ae61-4b0d-82a0-ad523cd15402 tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:05.934035 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8404a1-ae61-4b0d-82a0-ad523cd15402 tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:05.935435 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8404a1-ae61-4b0d-82a0-ad523cd15402 tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:05.936856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8404a1-ae61-4b0d-82a0-ad523cd15402 tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:05.962921 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a79968a-217b-4146-826c-41ff470c897d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:05.963603 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79968a-217b-4146-826c-41ff470c897d service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:05.963924 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79968a-217b-4146-826c-41ff470c897d service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:05.964166 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79968a-217b-4146-826c-41ff470c897d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:05.966050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4a79968a-217b-4146-826c-41ff470c897d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:05.978850 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 948/1875] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:05 2022] POST /identity/v3/projects => generated 353 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:05.978850 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8404a1-ae61-4b0d-82a0-ad523cd15402 tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:05.979951 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-be8404a1-ae61-4b0d-82a0-ad523cd15402 tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:06.044889 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a79968a-217b-4146-826c-41ff470c897d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:06.046625 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a79968a-217b-4146-826c-41ff470c897d service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:06.054055 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:06.062921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8404a1-ae61-4b0d-82a0-ad523cd15402 tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:06.063962 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 928/1876] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:06.065523 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412c08a-f0bc-4e78-87ef-8a3905a8ec32 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:06.065919 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412c08a-f0bc-4e78-87ef-8a3905a8ec32 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:06.066308 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412c08a-f0bc-4e78-87ef-8a3905a8ec32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:06.069394 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a9743-cd83-41af-8379-48e7b1587e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:06.070436 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a9743-cd83-41af-8379-48e7b1587e3d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:06.070726 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a9743-cd83-41af-8379-48e7b1587e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:06.071002 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a9743-cd83-41af-8379-48e7b1587e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:06.071740 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08a9743-cd83-41af-8379-48e7b1587e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:06.073544 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08a9743-cd83-41af-8379-48e7b1587e3d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:06.085044 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5412c08a-f0bc-4e78-87ef-8a3905a8ec32 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:06.086445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5412c08a-f0bc-4e78-87ef-8a3905a8ec32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:06.089714 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 949/1877] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:06.094265 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a9743-cd83-41af-8379-48e7b1587e3d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:06.102313 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 929/1878] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:06.106352 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5412c08a-f0bc-4e78-87ef-8a3905a8ec32 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:06.119577 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5412c08a-f0bc-4e78-87ef-8a3905a8ec32 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:06.127804 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236ba5c-dcbc-4ae5-b1d5-bda2bfbeefa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:06.129047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236ba5c-dcbc-4ae5-b1d5-bda2bfbeefa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:06.129415 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236ba5c-dcbc-4ae5-b1d5-bda2bfbeefa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:06.129799 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236ba5c-dcbc-4ae5-b1d5-bda2bfbeefa1 admin admin] PATH_INFO: `/v3/projects/8ee4156be29e4784bc71a50642807dbb/users/69f3459ed7344c2dbb465127dd662271/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:06.134093 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9236ba5c-dcbc-4ae5-b1d5-bda2bfbeefa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ee4156be29e4784bc71a50642807dbb, user_id=69f3459ed7344c2dbb465127dd662271, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:06.137238 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9236ba5c-dcbc-4ae5-b1d5-bda2bfbeefa1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:06.151780 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 930/1879] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:06 2022] PUT /identity/v3/projects/8ee4156be29e4784bc71a50642807dbb/users/69f3459ed7344c2dbb465127dd662271/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:06.152952 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236ba5c-dcbc-4ae5-b1d5-bda2bfbeefa1 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:06.162569 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9236ba5c-dcbc-4ae5-b1d5-bda2bfbeefa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:06.194477 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a9743-cd83-41af-8379-48e7b1587e3d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:06.217845 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:06.277276 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236ba5c-dcbc-4ae5-b1d5-bda2bfbeefa1 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:06.287444 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b1953-1e85-43a8-af2c-f9a162d63d7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:06.287444 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b1953-1e85-43a8-af2c-f9a162d63d7c service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:06.287737 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b1953-1e85-43a8-af2c-f9a162d63d7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:06.287997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b1953-1e85-43a8-af2c-f9a162d63d7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:06.319410 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a4cbd-9c90-429d-b23d-422a6d6b1dcc tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4af8704137834929a2b48e5d8a0659e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3c99cfd46b64ceeb40d464180fac8de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:06.320972 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a4cbd-9c90-429d-b23d-422a6d6b1dcc tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:06.321381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a4cbd-9c90-429d-b23d-422a6d6b1dcc tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:06.321810 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a4cbd-9c90-429d-b23d-422a6d6b1dcc tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:06.322678 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a4cbd-9c90-429d-b23d-422a6d6b1dcc tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:05:06.323135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a4cbd-9c90-429d-b23d-422a6d6b1dcc tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:06.324958 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a4cbd-9c90-429d-b23d-422a6d6b1dcc tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:06.341972 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 950/1880] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:05:06.343833 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d168245-0552-452a-8c45-1d1773f3b6b2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:06.344198 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d168245-0552-452a-8c45-1d1773f3b6b2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:06.349810 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d168245-0552-452a-8c45-1d1773f3b6b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:06.385473 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9d168245-0552-452a-8c45-1d1773f3b6b2 None None] MFA Rules not processed for user `69f3459ed7344c2dbb465127dd662271`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:06.387241 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9d168245-0552-452a-8c45-1d1773f3b6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:06.393874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69b1953-1e85-43a8-af2c-f9a162d63d7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:06.395547 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69b1953-1e85-43a8-af2c-f9a162d63d7c service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:06.410007 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 931/1881] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:06.411241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b1953-1e85-43a8-af2c-f9a162d63d7c service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:06.424884 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b1953-1e85-43a8-af2c-f9a162d63d7c service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:06.431990 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55648ab-b81b-4866-85cf-5688f64f576c tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4af8704137834929a2b48e5d8a0659e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3c99cfd46b64ceeb40d464180fac8de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:06.433124 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55648ab-b81b-4866-85cf-5688f64f576c tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:06.433704 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55648ab-b81b-4866-85cf-5688f64f576c tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:06.434129 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55648ab-b81b-4866-85cf-5688f64f576c tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] PATH_INFO: `/v3/projects/5771702931974163a087845a17beb571` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:06.435616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55648ab-b81b-4866-85cf-5688f64f576c tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5771702931974163a087845a17beb571)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:06.438141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55648ab-b81b-4866-85cf-5688f64f576c tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:06.463372 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 951/1882] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:06 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:06.479723 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d168245-0552-452a-8c45-1d1773f3b6b2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:06.481880 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9d168245-0552-452a-8c45-1d1773f3b6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:06.517511 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f55648ab-b81b-4866-85cf-5688f64f576c tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5771702931974163a087845a17beb571', 'request_id': 'req-f55648ab-b81b-4866-85cf-5688f64f576c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:06.527170 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f55648ab-b81b-4866-85cf-5688f64f576c tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5771702931974163a087845a17beb571', 'request_id': 'req-f55648ab-b81b-4866-85cf-5688f64f576c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:06.538635 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f55648ab-b81b-4866-85cf-5688f64f576c tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5771702931974163a087845a17beb571', 'request_id': 'req-f55648ab-b81b-4866-85cf-5688f64f576c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:06.580760 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f55648ab-b81b-4866-85cf-5688f64f576c tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] The token cache is being invalidate because project 5771702931974163a087845a17beb571 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:06.581201 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f55648ab-b81b-4866-85cf-5688f64f576c tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:06.582794 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f55648ab-b81b-4866-85cf-5688f64f576c tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:06.584129 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d168245-0552-452a-8c45-1d1773f3b6b2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:06.591177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 932/1883] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:06 2022] DELETE /identity/v3/projects/5771702931974163a087845a17beb571 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:06.591177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55648ab-b81b-4866-85cf-5688f64f576c tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:06.594075 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f55648ab-b81b-4866-85cf-5688f64f576c tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:06.639979 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:06.672493 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee67b93-3a52-4fcd-a5cc-5bb0cb1a3663 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:06.673319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee67b93-3a52-4fcd-a5cc-5bb0cb1a3663 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:06.673630 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee67b93-3a52-4fcd-a5cc-5bb0cb1a3663 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:06.673918 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee67b93-3a52-4fcd-a5cc-5bb0cb1a3663 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:06.677146 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3ee67b93-3a52-4fcd-a5cc-5bb0cb1a3663 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:06.689504 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:06.785441 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee67b93-3a52-4fcd-a5cc-5bb0cb1a3663 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:06.787162 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee67b93-3a52-4fcd-a5cc-5bb0cb1a3663 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:06.794985 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55648ab-b81b-4866-85cf-5688f64f576c tempest-VolumeQuotaClassesTest-675427030 tempest-VolumeQuotaClassesTest-675427030-project-admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:06.804959 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 952/1884] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:06 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:06.825155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e070321-4361-4edc-bdcb-ddda7b9222cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:06.826722 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e070321-4361-4edc-bdcb-ddda7b9222cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:06.826896 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e070321-4361-4edc-bdcb-ddda7b9222cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:06.827181 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e070321-4361-4edc-bdcb-ddda7b9222cf admin admin] PATH_INFO: `/v3/projects/5ba022c31e33473082273f1e1ea6e0f1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:06.828623 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e070321-4361-4edc-bdcb-ddda7b9222cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ba022c31e33473082273f1e1ea6e0f1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:06.836140 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e070321-4361-4edc-bdcb-ddda7b9222cf admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:06.913414 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee67b93-3a52-4fcd-a5cc-5bb0cb1a3663 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:06.916043 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3ee67b93-3a52-4fcd-a5cc-5bb0cb1a3663 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:06.938504 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3e070321-4361-4edc-bdcb-ddda7b9222cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ba022c31e33473082273f1e1ea6e0f1', 'request_id': 'req-3e070321-4361-4edc-bdcb-ddda7b9222cf'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:06.942594 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee67b93-3a52-4fcd-a5cc-5bb0cb1a3663 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:06.947201 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3e070321-4361-4edc-bdcb-ddda7b9222cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ba022c31e33473082273f1e1ea6e0f1', 'request_id': 'req-3e070321-4361-4edc-bdcb-ddda7b9222cf'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:06.958963 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3e070321-4361-4edc-bdcb-ddda7b9222cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ba022c31e33473082273f1e1ea6e0f1', 'request_id': 'req-3e070321-4361-4edc-bdcb-ddda7b9222cf'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:06.973746 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3e070321-4361-4edc-bdcb-ddda7b9222cf admin admin] The token cache is being invalidate because project 5ba022c31e33473082273f1e1ea6e0f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:06.974038 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3e070321-4361-4edc-bdcb-ddda7b9222cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:06.974866 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3e070321-4361-4edc-bdcb-ddda7b9222cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:06.977616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 933/1885] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:06 2022] DELETE /identity/v3/projects/5ba022c31e33473082273f1e1ea6e0f1 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:06.981721 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e070321-4361-4edc-bdcb-ddda7b9222cf admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:06.985943 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3e070321-4361-4edc-bdcb-ddda7b9222cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:07.086517 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10049afc-4369-4479-b4e7-bac230d37b31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:07.087197 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10049afc-4369-4479-b4e7-bac230d37b31 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:07.087481 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10049afc-4369-4479-b4e7-bac230d37b31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:07.087759 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10049afc-4369-4479-b4e7-bac230d37b31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:07.091726 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-10049afc-4369-4479-b4e7-bac230d37b31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:07.130697 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e070321-4361-4edc-bdcb-ddda7b9222cf admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:07.150265 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59cc2f-a8a7-48a1-a0eb-3df53a3954ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:07.151521 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59cc2f-a8a7-48a1-a0eb-3df53a3954ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:07.152121 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59cc2f-a8a7-48a1-a0eb-3df53a3954ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:07.153025 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59cc2f-a8a7-48a1-a0eb-3df53a3954ad admin admin] PATH_INFO: `/v3/users/31548b3ce8f54284803098ca8cd4c36f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:07.155329 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac59cc2f-a8a7-48a1-a0eb-3df53a3954ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=31548b3ce8f54284803098ca8cd4c36f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:07.158095 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac59cc2f-a8a7-48a1-a0eb-3df53a3954ad admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:07.200263 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10049afc-4369-4479-b4e7-bac230d37b31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:07.202548 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10049afc-4369-4479-b4e7-bac230d37b31 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:07.214134 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 953/1886] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 16:05:07.225438 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10049afc-4369-4479-b4e7-bac230d37b31 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:07.232883 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-10049afc-4369-4479-b4e7-bac230d37b31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:07.266870 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:07.269079 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ac59cc2f-a8a7-48a1-a0eb-3df53a3954ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31548b3ce8f54284803098ca8cd4c36f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:07.269933 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ac59cc2f-a8a7-48a1-a0eb-3df53a3954ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31548b3ce8f54284803098ca8cd4c36f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:07.270558 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ac59cc2f-a8a7-48a1-a0eb-3df53a3954ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31548b3ce8f54284803098ca8cd4c36f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:07.280854 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ac59cc2f-a8a7-48a1-a0eb-3df53a3954ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31548b3ce8f54284803098ca8cd4c36f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:07.289958 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10049afc-4369-4479-b4e7-bac230d37b31 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:07.294028 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ac59cc2f-a8a7-48a1-a0eb-3df53a3954ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31548b3ce8f54284803098ca8cd4c36f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:07.295961 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765da2e-8e6b-452a-afac-36e551244463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:07.296898 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765da2e-8e6b-452a-afac-36e551244463 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:07.297178 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765da2e-8e6b-452a-afac-36e551244463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:07.297497 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765da2e-8e6b-452a-afac-36e551244463 admin admin] PATH_INFO: `/v3/users/ccedd4a8370e4c51a0be88be92458237` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:07.298772 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765da2e-8e6b-452a-afac-36e551244463 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccedd4a8370e4c51a0be88be92458237)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:07.301016 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765da2e-8e6b-452a-afac-36e551244463 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:07.345865 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 934/1887] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:06 2022] DELETE /identity/v3/users/31548b3ce8f54284803098ca8cd4c36f => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:05:07.395932 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5765da2e-8e6b-452a-afac-36e551244463 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccedd4a8370e4c51a0be88be92458237'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:07.397431 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5765da2e-8e6b-452a-afac-36e551244463 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccedd4a8370e4c51a0be88be92458237'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:07.399344 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5765da2e-8e6b-452a-afac-36e551244463 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccedd4a8370e4c51a0be88be92458237'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:07.417124 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:07.421949 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5765da2e-8e6b-452a-afac-36e551244463 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccedd4a8370e4c51a0be88be92458237'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:07.436603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5765da2e-8e6b-452a-afac-36e551244463 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccedd4a8370e4c51a0be88be92458237'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:07.501055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 954/1888] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:07 2022] DELETE /identity/v3/users/ccedd4a8370e4c51a0be88be92458237 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:07.506631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531a114-3079-4870-a80e-da9b38802bd9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:07.506631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531a114-3079-4870-a80e-da9b38802bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:07.506994 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531a114-3079-4870-a80e-da9b38802bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:07.557747 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e531a114-3079-4870-a80e-da9b38802bd9 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:07.561924 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e531a114-3079-4870-a80e-da9b38802bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:07.684077 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 935/1889] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:07.694559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765da2e-8e6b-452a-afac-36e551244463 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:07.696236 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5765da2e-8e6b-452a-afac-36e551244463 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:07.813006 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765da2e-8e6b-452a-afac-36e551244463 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:07.826485 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644351b4-f388-4a84-9396-34f48793d6e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:07.828002 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644351b4-f388-4a84-9396-34f48793d6e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:07.828002 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644351b4-f388-4a84-9396-34f48793d6e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:07.828002 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644351b4-f388-4a84-9396-34f48793d6e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:07.850725 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644351b4-f388-4a84-9396-34f48793d6e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:07.855771 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644351b4-f388-4a84-9396-34f48793d6e3 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:07.872288 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 955/1890] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:07.883427 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531a114-3079-4870-a80e-da9b38802bd9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:07.885150 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e531a114-3079-4870-a80e-da9b38802bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:07.899401 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531a114-3079-4870-a80e-da9b38802bd9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:07.905174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb2868-0833-4b54-8723-422bb2e0d281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:07.906239 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb2868-0833-4b54-8723-422bb2e0d281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:07.906686 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb2868-0833-4b54-8723-422bb2e0d281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:07.907074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb2868-0833-4b54-8723-422bb2e0d281 admin admin] PATH_INFO: `/v3/projects/489be9911f7a4eb9be1e60a073e13bce` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:07.908545 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bb2868-0833-4b54-8723-422bb2e0d281 admin admin] RBAC: Authorizing `identity:delete_project(project_id=489be9911f7a4eb9be1e60a073e13bce)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:07.910904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bb2868-0833-4b54-8723-422bb2e0d281 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:07.998396 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a8bb2868-0833-4b54-8723-422bb2e0d281 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '489be9911f7a4eb9be1e60a073e13bce', 'request_id': 'req-a8bb2868-0833-4b54-8723-422bb2e0d281'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:08.012888 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a8bb2868-0833-4b54-8723-422bb2e0d281 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '489be9911f7a4eb9be1e60a073e13bce', 'request_id': 'req-a8bb2868-0833-4b54-8723-422bb2e0d281'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:08.023320 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a8bb2868-0833-4b54-8723-422bb2e0d281 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '489be9911f7a4eb9be1e60a073e13bce', 'request_id': 'req-a8bb2868-0833-4b54-8723-422bb2e0d281'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:08.035655 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a8bb2868-0833-4b54-8723-422bb2e0d281 admin admin] The token cache is being invalidate because project 489be9911f7a4eb9be1e60a073e13bce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:08.035655 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a8bb2868-0833-4b54-8723-422bb2e0d281 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:08.036428 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a8bb2868-0833-4b54-8723-422bb2e0d281 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:08.040974 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 936/1891] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:07 2022] DELETE /identity/v3/projects/489be9911f7a4eb9be1e60a073e13bce => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:08.042388 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644351b4-f388-4a84-9396-34f48793d6e3 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:08.055870 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644351b4-f388-4a84-9396-34f48793d6e3 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:08.141107 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2461a0-5eb0-4143-893d-63bfeb9ffcb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:08.142145 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2461a0-5eb0-4143-893d-63bfeb9ffcb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:08.151363 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2461a0-5eb0-4143-893d-63bfeb9ffcb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:08.151363 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2461a0-5eb0-4143-893d-63bfeb9ffcb1 admin admin] PATH_INFO: `/v3/users/5d830eeb1bc44e5bbd4e160ad7003725` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:08.151363 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2461a0-5eb0-4143-893d-63bfeb9ffcb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d830eeb1bc44e5bbd4e160ad7003725)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:08.151363 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2461a0-5eb0-4143-893d-63bfeb9ffcb1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:08.205032 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2e2461a0-5eb0-4143-893d-63bfeb9ffcb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d830eeb1bc44e5bbd4e160ad7003725'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:08.205994 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2e2461a0-5eb0-4143-893d-63bfeb9ffcb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d830eeb1bc44e5bbd4e160ad7003725'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:08.206777 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2e2461a0-5eb0-4143-893d-63bfeb9ffcb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d830eeb1bc44e5bbd4e160ad7003725'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:08.216320 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2e2461a0-5eb0-4143-893d-63bfeb9ffcb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d830eeb1bc44e5bbd4e160ad7003725'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:08.230589 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb2868-0833-4b54-8723-422bb2e0d281 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:08.232397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2e2461a0-5eb0-4143-893d-63bfeb9ffcb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d830eeb1bc44e5bbd4e160ad7003725'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:08.232730 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a8bb2868-0833-4b54-8723-422bb2e0d281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:08.249036 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb2868-0833-4b54-8723-422bb2e0d281 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:08.254195 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f1f95-cd28-450c-93a7-b9fef62dacd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:08.255261 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f1f95-cd28-450c-93a7-b9fef62dacd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:08.255670 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f1f95-cd28-450c-93a7-b9fef62dacd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:08.255892 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f1f95-cd28-450c-93a7-b9fef62dacd9 admin admin] PATH_INFO: `/v3/projects/a4de7cf8aa104c5aa135f60467a5ba10` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:08.257952 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f1f95-cd28-450c-93a7-b9fef62dacd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4de7cf8aa104c5aa135f60467a5ba10)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:08.258580 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:08.262491 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f1f95-cd28-450c-93a7-b9fef62dacd9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:08.273108 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 956/1892] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:08 2022] DELETE /identity/v3/users/5d830eeb1bc44e5bbd4e160ad7003725 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:08.274977 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2461a0-5eb0-4143-893d-63bfeb9ffcb1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:08.279494 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2e2461a0-5eb0-4143-893d-63bfeb9ffcb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:08.325644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:08.334801 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-878f1f95-cd28-450c-93a7-b9fef62dacd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4de7cf8aa104c5aa135f60467a5ba10', 'request_id': 'req-878f1f95-cd28-450c-93a7-b9fef62dacd9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:08.342806 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-878f1f95-cd28-450c-93a7-b9fef62dacd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4de7cf8aa104c5aa135f60467a5ba10', 'request_id': 'req-878f1f95-cd28-450c-93a7-b9fef62dacd9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:08.351628 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-878f1f95-cd28-450c-93a7-b9fef62dacd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4de7cf8aa104c5aa135f60467a5ba10', 'request_id': 'req-878f1f95-cd28-450c-93a7-b9fef62dacd9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:08.363583 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-878f1f95-cd28-450c-93a7-b9fef62dacd9 admin admin] The token cache is being invalidate because project a4de7cf8aa104c5aa135f60467a5ba10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:08.363911 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-878f1f95-cd28-450c-93a7-b9fef62dacd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:08.364934 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-878f1f95-cd28-450c-93a7-b9fef62dacd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:08.369740 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 937/1893] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:08 2022] DELETE /identity/v3/projects/a4de7cf8aa104c5aa135f60467a5ba10 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:05:08.372362 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f681b76-17a5-406b-b41a-f789bb267700 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:08.372662 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2461a0-5eb0-4143-893d-63bfeb9ffcb1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:08.372768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f681b76-17a5-406b-b41a-f789bb267700 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:08.373381 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f681b76-17a5-406b-b41a-f789bb267700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:08.392651 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5f681b76-17a5-406b-b41a-f789bb267700 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:08.393916 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5f681b76-17a5-406b-b41a-f789bb267700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:08.456176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498934e1-4dd2-47bc-84ab-2ed76647e7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:08.457780 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498934e1-4dd2-47bc-84ab-2ed76647e7ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:08.458096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498934e1-4dd2-47bc-84ab-2ed76647e7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:08.458689 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498934e1-4dd2-47bc-84ab-2ed76647e7ea admin admin] PATH_INFO: `/v3/projects/706be92d077f412bbf64e7f0e674ad6b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:08.461121 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498934e1-4dd2-47bc-84ab-2ed76647e7ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=706be92d077f412bbf64e7f0e674ad6b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:08.463371 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498934e1-4dd2-47bc-84ab-2ed76647e7ea admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:08.473341 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 938/1894] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:05:08.482319 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7beb63e-335f-4103-958f-dae4c780fdb7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:08.482319 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7beb63e-335f-4103-958f-dae4c780fdb7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:08.482576 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7beb63e-335f-4103-958f-dae4c780fdb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:08.501850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c7beb63e-335f-4103-958f-dae4c780fdb7 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:08.503261 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c7beb63e-335f-4103-958f-dae4c780fdb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:08.519041 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 939/1895] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:05:08.524511 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7beb63e-335f-4103-958f-dae4c780fdb7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:08.531000 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-498934e1-4dd2-47bc-84ab-2ed76647e7ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '706be92d077f412bbf64e7f0e674ad6b', 'request_id': 'req-498934e1-4dd2-47bc-84ab-2ed76647e7ea'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:08.536546 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7beb63e-335f-4103-958f-dae4c780fdb7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:08.541291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-498934e1-4dd2-47bc-84ab-2ed76647e7ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '706be92d077f412bbf64e7f0e674ad6b', 'request_id': 'req-498934e1-4dd2-47bc-84ab-2ed76647e7ea'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:08.549663 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-498934e1-4dd2-47bc-84ab-2ed76647e7ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '706be92d077f412bbf64e7f0e674ad6b', 'request_id': 'req-498934e1-4dd2-47bc-84ab-2ed76647e7ea'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:08.576441 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-498934e1-4dd2-47bc-84ab-2ed76647e7ea admin admin] The token cache is being invalidate because project 706be92d077f412bbf64e7f0e674ad6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:08.576841 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-498934e1-4dd2-47bc-84ab-2ed76647e7ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:08.577982 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-498934e1-4dd2-47bc-84ab-2ed76647e7ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:08.580255 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 957/1896] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:08 2022] DELETE /identity/v3/projects/706be92d077f412bbf64e7f0e674ad6b => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:08.586435 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498934e1-4dd2-47bc-84ab-2ed76647e7ea admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:08.595019 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-498934e1-4dd2-47bc-84ab-2ed76647e7ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:08.612869 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50bf9af-1517-4886-8abd-15fc7ada53b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:08.613474 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50bf9af-1517-4886-8abd-15fc7ada53b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:08.613924 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50bf9af-1517-4886-8abd-15fc7ada53b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:08.614085 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50bf9af-1517-4886-8abd-15fc7ada53b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:08.614829 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50bf9af-1517-4886-8abd-15fc7ada53b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:05:08.615686 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50bf9af-1517-4886-8abd-15fc7ada53b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:08.617836 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50bf9af-1517-4886-8abd-15fc7ada53b4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:08.634793 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 940/1897] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:05:08.635813 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50bf9af-1517-4886-8abd-15fc7ada53b4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:08.639793 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e50bf9af-1517-4886-8abd-15fc7ada53b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:08.655207 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50bf9af-1517-4886-8abd-15fc7ada53b4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:08.662340 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:08.663285 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c541435-0a29-4e55-8921-bb504f947171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:08.664269 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c541435-0a29-4e55-8921-bb504f947171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:08.664784 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c541435-0a29-4e55-8921-bb504f947171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:08.665011 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c541435-0a29-4e55-8921-bb504f947171 admin admin] PATH_INFO: `/v3/projects/05b45b43b383499da00bff9eabb1f5a8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:08.666382 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c541435-0a29-4e55-8921-bb504f947171 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05b45b43b383499da00bff9eabb1f5a8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:08.668584 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c541435-0a29-4e55-8921-bb504f947171 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:08.702600 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498934e1-4dd2-47bc-84ab-2ed76647e7ea admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:08.708441 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b25c910-31d4-4b58-8138-cb8a3d537a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:08.709483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25c910-31d4-4b58-8138-cb8a3d537a9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:08.709800 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25c910-31d4-4b58-8138-cb8a3d537a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:08.710123 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25c910-31d4-4b58-8138-cb8a3d537a9b admin admin] PATH_INFO: `/v3/users/4af8704137834929a2b48e5d8a0659e6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:08.711435 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b25c910-31d4-4b58-8138-cb8a3d537a9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4af8704137834929a2b48e5d8a0659e6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:08.713365 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b25c910-31d4-4b58-8138-cb8a3d537a9b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:08.738837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2c541435-0a29-4e55-8921-bb504f947171 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05b45b43b383499da00bff9eabb1f5a8', 'request_id': 'req-2c541435-0a29-4e55-8921-bb504f947171'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:08.744825 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:08.755602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2c541435-0a29-4e55-8921-bb504f947171 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05b45b43b383499da00bff9eabb1f5a8', 'request_id': 'req-2c541435-0a29-4e55-8921-bb504f947171'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:08.775761 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2c541435-0a29-4e55-8921-bb504f947171 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05b45b43b383499da00bff9eabb1f5a8', 'request_id': 'req-2c541435-0a29-4e55-8921-bb504f947171'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:08.801439 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2c541435-0a29-4e55-8921-bb504f947171 admin admin] The token cache is being invalidate because project 05b45b43b383499da00bff9eabb1f5a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:08.801836 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2c541435-0a29-4e55-8921-bb504f947171 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:08.802453 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2c541435-0a29-4e55-8921-bb504f947171 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:08.805129 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 941/1898] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:08 2022] DELETE /identity/v3/projects/05b45b43b383499da00bff9eabb1f5a8 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:08.808928 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7b25c910-31d4-4b58-8138-cb8a3d537a9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4af8704137834929a2b48e5d8a0659e6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:08.811068 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c541435-0a29-4e55-8921-bb504f947171 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:08.811814 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2c541435-0a29-4e55-8921-bb504f947171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:08.812652 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7b25c910-31d4-4b58-8138-cb8a3d537a9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4af8704137834929a2b48e5d8a0659e6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:08.814756 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7b25c910-31d4-4b58-8138-cb8a3d537a9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4af8704137834929a2b48e5d8a0659e6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:08.828803 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7b25c910-31d4-4b58-8138-cb8a3d537a9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4af8704137834929a2b48e5d8a0659e6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:08.842775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7b25c910-31d4-4b58-8138-cb8a3d537a9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4af8704137834929a2b48e5d8a0659e6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:08.892058 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 958/1899] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:08 2022] DELETE /identity/v3/users/4af8704137834929a2b48e5d8a0659e6 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:08.892058 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84df839-6cb5-49a0-99a4-b36f25a9edbb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:08.892058 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84df839-6cb5-49a0-99a4-b36f25a9edbb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:08.892524 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84df839-6cb5-49a0-99a4-b36f25a9edbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:08.926544 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c541435-0a29-4e55-8921-bb504f947171 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:08.946370 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d84df839-6cb5-49a0-99a4-b36f25a9edbb None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:08.947616 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d84df839-6cb5-49a0-99a4-b36f25a9edbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:09.046052 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c8995-642e-4f1c-b44b-c83fc6a57625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:09.048469 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c8995-642e-4f1c-b44b-c83fc6a57625 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:09.049810 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c8995-642e-4f1c-b44b-c83fc6a57625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:09.051642 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c8995-642e-4f1c-b44b-c83fc6a57625 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:09.053466 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c8995-642e-4f1c-b44b-c83fc6a57625 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:09.059419 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c8995-642e-4f1c-b44b-c83fc6a57625 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:09.115435 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 959/1900] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:09.123267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac33b4-ccc7-4c66-bf35-1e4a52353da0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:09.123561 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac33b4-ccc7-4c66-bf35-1e4a52353da0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:09.123835 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac33b4-ccc7-4c66-bf35-1e4a52353da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:09.160823 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 942/1901] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:08 2022] POST /identity/v3/projects => generated 389 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:09.167459 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c8995-642e-4f1c-b44b-c83fc6a57625 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:09.170111 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fa3c8995-642e-4f1c-b44b-c83fc6a57625 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:09.186216 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-28ac33b4-ccc7-4c66-bf35-1e4a52353da0 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:09.186216 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-28ac33b4-ccc7-4c66-bf35-1e4a52353da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:09.336388 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c8995-642e-4f1c-b44b-c83fc6a57625 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:09.342428 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c920d543-11db-42b3-8be4-ade62278b2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:09.344241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920d543-11db-42b3-8be4-ade62278b2fe admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:09.344241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920d543-11db-42b3-8be4-ade62278b2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:09.344241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920d543-11db-42b3-8be4-ade62278b2fe admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:09.344899 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c920d543-11db-42b3-8be4-ade62278b2fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:09.347584 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c920d543-11db-42b3-8be4-ade62278b2fe admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:09.354363 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:09.391790 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 943/1902] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:09 2022] POST /identity/v3/users => generated 295 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:05:09.391790 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33523f94-52fb-4075-b522-bb166ee59a3c None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:09.392240 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33523f94-52fb-4075-b522-bb166ee59a3c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:09.393926 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33523f94-52fb-4075-b522-bb166ee59a3c None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:09.396925 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 960/1903] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:09.398511 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac33b4-ccc7-4c66-bf35-1e4a52353da0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:09.400388 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 944/1904] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jul 21 16:05:09 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 16:05:09.402474 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33523f94-52fb-4075-b522-bb166ee59a3c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:09.418674 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:09.421414 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac33b4-ccc7-4c66-bf35-1e4a52353da0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:09.435240 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33523f94-52fb-4075-b522-bb166ee59a3c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:09.445763 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b1717-12a7-4301-9b44-267f809b8305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:09.446468 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b1717-12a7-4301-9b44-267f809b8305 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:09.446767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b1717-12a7-4301-9b44-267f809b8305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:09.447051 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b1717-12a7-4301-9b44-267f809b8305 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:09.447804 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1b1717-12a7-4301-9b44-267f809b8305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:09.451088 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc276e14-22fc-4199-89ff-581f1797e721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:09.451774 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc276e14-22fc-4199-89ff-581f1797e721 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:09.452063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc276e14-22fc-4199-89ff-581f1797e721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:09.452349 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc276e14-22fc-4199-89ff-581f1797e721 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:09.453089 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc276e14-22fc-4199-89ff-581f1797e721 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:05:09.453540 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc276e14-22fc-4199-89ff-581f1797e721 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:09.455422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1b1717-12a7-4301-9b44-267f809b8305 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:09.456233 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc276e14-22fc-4199-89ff-581f1797e721 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:09.478990 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 961/1905] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:09.481058 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7525b-6673-42dc-b5ea-1793429a5738 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:09.481415 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7525b-6673-42dc-b5ea-1793429a5738 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:09.481947 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7525b-6673-42dc-b5ea-1793429a5738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:09.486785 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 945/1906] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:05:09.491007 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc276e14-22fc-4199-89ff-581f1797e721 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:09.510975 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc276e14-22fc-4199-89ff-581f1797e721 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:09.520205 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e0f7525b-6673-42dc-b5ea-1793429a5738 None None] MFA Rules not processed for user `092553a86f8848dba8c7271f5c0b2c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:09.522223 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e0f7525b-6673-42dc-b5ea-1793429a5738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:09.524890 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19854e2f-61db-4451-b3d0-e7a6e4757eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:09.526104 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19854e2f-61db-4451-b3d0-e7a6e4757eb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:09.526104 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19854e2f-61db-4451-b3d0-e7a6e4757eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:09.526299 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19854e2f-61db-4451-b3d0-e7a6e4757eb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:09.526925 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19854e2f-61db-4451-b3d0-e7a6e4757eb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:09.528552 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19854e2f-61db-4451-b3d0-e7a6e4757eb6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:09.546007 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 946/1907] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:09.546852 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19854e2f-61db-4451-b3d0-e7a6e4757eb6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:09.589673 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19854e2f-61db-4451-b3d0-e7a6e4757eb6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:09.595713 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956b6f3-cb7b-42f4-aa66-81557d83ef97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:09.596388 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956b6f3-cb7b-42f4-aa66-81557d83ef97 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:09.596672 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956b6f3-cb7b-42f4-aa66-81557d83ef97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:09.597060 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956b6f3-cb7b-42f4-aa66-81557d83ef97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:09.597825 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8956b6f3-cb7b-42f4-aa66-81557d83ef97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:09.602108 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8956b6f3-cb7b-42f4-aa66-81557d83ef97 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:09.661497 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 947/1908] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:09 2022] POST /identity/v3/projects => generated 379 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:09.661497 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956b6f3-cb7b-42f4-aa66-81557d83ef97 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:09.691927 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 962/1909] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jul 21 16:05:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:09.697506 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7525b-6673-42dc-b5ea-1793429a5738 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:09.713373 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7525b-6673-42dc-b5ea-1793429a5738 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:09.768556 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956b6f3-cb7b-42f4-aa66-81557d83ef97 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:09.775864 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf5db9-cda1-475c-be57-18e57f3af88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:09.777137 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf5db9-cda1-475c-be57-18e57f3af88d admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:09.778030 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf5db9-cda1-475c-be57-18e57f3af88d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:09.778440 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf5db9-cda1-475c-be57-18e57f3af88d admin admin] PATH_INFO: `/v3/projects/29c95ae60ca14ab897570df35995ab15/users/69bf76277d6e4577a826e7ec3dcfe708/roles/54195e660fc349978b39d5794765a095` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:09.788252 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35e9ce-42b2-4319-9cdb-4640d812302c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:09.789734 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35e9ce-42b2-4319-9cdb-4640d812302c admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:09.790396 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35e9ce-42b2-4319-9cdb-4640d812302c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:09.790786 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35e9ce-42b2-4319-9cdb-4640d812302c admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:09.791551 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35e9ce-42b2-4319-9cdb-4640d812302c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:09.793861 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35e9ce-42b2-4319-9cdb-4640d812302c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:09.802078 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf5db9-cda1-475c-be57-18e57f3af88d admin admin] RBAC: Authorizing `identity:create_grant(project_id=29c95ae60ca14ab897570df35995ab15, user_id=69bf76277d6e4577a826e7ec3dcfe708, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:09.805706 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf5db9-cda1-475c-be57-18e57f3af88d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:09.822333 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 948/1910] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:09 2022] PUT /identity/v3/projects/29c95ae60ca14ab897570df35995ab15/users/69bf76277d6e4577a826e7ec3dcfe708/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:09.823169 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf5db9-cda1-475c-be57-18e57f3af88d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:09.825527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 963/1911] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:09 2022] POST /identity/v3/users => generated 290 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:09.826435 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35e9ce-42b2-4319-9cdb-4640d812302c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:09.828079 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-eb35e9ce-42b2-4319-9cdb-4640d812302c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:09.842798 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-49cf5db9-cda1-475c-be57-18e57f3af88d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:09.914347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:09.938721 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35e9ce-42b2-4319-9cdb-4640d812302c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:09.943297 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf5db9-cda1-475c-be57-18e57f3af88d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:09.948529 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf57a1-2a0c-4e2f-9df2-a7b2134a18e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:09.949679 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf57a1-2a0c-4e2f-9df2-a7b2134a18e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:09.950461 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf57a1-2a0c-4e2f-9df2-a7b2134a18e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:09.950825 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf57a1-2a0c-4e2f-9df2-a7b2134a18e5 admin admin] PATH_INFO: `/v3/projects/b3c99cfd46b64ceeb40d464180fac8de` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:09.952678 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf57a1-2a0c-4e2f-9df2-a7b2134a18e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3c99cfd46b64ceeb40d464180fac8de)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:09.955711 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf57a1-2a0c-4e2f-9df2-a7b2134a18e5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:10.004044 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:10.011765 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5cbf57a1-2a0c-4e2f-9df2-a7b2134a18e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3c99cfd46b64ceeb40d464180fac8de', 'request_id': 'req-5cbf57a1-2a0c-4e2f-9df2-a7b2134a18e5'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:10.014113 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e0428-869d-49a5-ac16-ff376761d3d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:10.014780 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e0428-869d-49a5-ac16-ff376761d3d3 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:10.015065 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e0428-869d-49a5-ac16-ff376761d3d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:10.015343 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e0428-869d-49a5-ac16-ff376761d3d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:10.018237 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5cbf57a1-2a0c-4e2f-9df2-a7b2134a18e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3c99cfd46b64ceeb40d464180fac8de', 'request_id': 'req-5cbf57a1-2a0c-4e2f-9df2-a7b2134a18e5'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:10.025559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e0428-869d-49a5-ac16-ff376761d3d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:10.026482 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5cbf57a1-2a0c-4e2f-9df2-a7b2134a18e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3c99cfd46b64ceeb40d464180fac8de', 'request_id': 'req-5cbf57a1-2a0c-4e2f-9df2-a7b2134a18e5'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:10.027372 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e0428-869d-49a5-ac16-ff376761d3d3 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:10.037099 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5cbf57a1-2a0c-4e2f-9df2-a7b2134a18e5 admin admin] The token cache is being invalidate because project b3c99cfd46b64ceeb40d464180fac8de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:10.037404 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5cbf57a1-2a0c-4e2f-9df2-a7b2134a18e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:10.038106 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5cbf57a1-2a0c-4e2f-9df2-a7b2134a18e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:10.039743 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 949/1912] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:09 2022] DELETE /identity/v3/projects/b3c99cfd46b64ceeb40d464180fac8de => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:10.043985 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf57a1-2a0c-4e2f-9df2-a7b2134a18e5 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:10.044104 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 964/1913] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:10.044104 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6078934-feb5-44fd-baea-101ca9fdc507 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:10.044104 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6078934-feb5-44fd-baea-101ca9fdc507 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:10.044104 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6078934-feb5-44fd-baea-101ca9fdc507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:10.045197 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5cbf57a1-2a0c-4e2f-9df2-a7b2134a18e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:10.072313 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e6078934-feb5-44fd-baea-101ca9fdc507 None None] MFA Rules not processed for user `69bf76277d6e4577a826e7ec3dcfe708`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:10.073540 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e6078934-feb5-44fd-baea-101ca9fdc507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:10.120948 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf57a1-2a0c-4e2f-9df2-a7b2134a18e5 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:10.125685 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877bb161-904b-4abd-9d17-a88de98e6606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:10.126425 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877bb161-904b-4abd-9d17-a88de98e6606 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:10.126646 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877bb161-904b-4abd-9d17-a88de98e6606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:10.126928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877bb161-904b-4abd-9d17-a88de98e6606 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:10.127701 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877bb161-904b-4abd-9d17-a88de98e6606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:10.129345 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877bb161-904b-4abd-9d17-a88de98e6606 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:10.139852 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 950/1914] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:10.141211 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffccf9f-3dd5-45f9-86b1-423b71c9c4d7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:10.141425 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffccf9f-3dd5-45f9-86b1-423b71c9c4d7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:10.141732 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffccf9f-3dd5-45f9-86b1-423b71c9c4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:10.158673 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 965/1915] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:10.159855 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6078934-feb5-44fd-baea-101ca9fdc507 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:10.161173 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0ffccf9f-3dd5-45f9-86b1-423b71c9c4d7 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:10.162411 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0ffccf9f-3dd5-45f9-86b1-423b71c9c4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:10.170648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 951/1916] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:10.176807 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffccf9f-3dd5-45f9-86b1-423b71c9c4d7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:10.176807 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0ffccf9f-3dd5-45f9-86b1-423b71c9c4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:10.177008 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6078934-feb5-44fd-baea-101ca9fdc507 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:10.183141 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3520930-225d-40cb-a5d3-e6688a6799f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:10.183974 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3520930-225d-40cb-a5d3-e6688a6799f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:10.184274 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3520930-225d-40cb-a5d3-e6688a6799f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:10.184580 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3520930-225d-40cb-a5d3-e6688a6799f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:10.185391 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3520930-225d-40cb-a5d3-e6688a6799f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:10.187259 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3520930-225d-40cb-a5d3-e6688a6799f8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:10.188888 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffccf9f-3dd5-45f9-86b1-423b71c9c4d7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:10.193559 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d965dba-1e65-4c32-a100-83d49405cc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:10.194243 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d965dba-1e65-4c32-a100-83d49405cc58 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:10.194706 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d965dba-1e65-4c32-a100-83d49405cc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:10.194895 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d965dba-1e65-4c32-a100-83d49405cc58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:10.195556 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d965dba-1e65-4c32-a100-83d49405cc58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:10.197686 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d965dba-1e65-4c32-a100-83d49405cc58 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:10.198616 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 966/1917] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:10.202943 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ec283-ac66-46ea-99c7-ced87b60c7f5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:10.203306 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ec283-ac66-46ea-99c7-ced87b60c7f5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:10.203670 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ec283-ac66-46ea-99c7-ced87b60c7f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:10.227527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-512ec283-ac66-46ea-99c7-ced87b60c7f5 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:10.228489 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 952/1918] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:10 2022] POST /identity/v3/projects => generated 389 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:10.229374 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d965dba-1e65-4c32-a100-83d49405cc58 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:10.232833 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-512ec283-ac66-46ea-99c7-ced87b60c7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:10.337686 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d965dba-1e65-4c32-a100-83d49405cc58 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:10.344390 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe5538-970f-4f91-aa01-e5703c266fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:10.345422 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe5538-970f-4f91-aa01-e5703c266fb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:10.345744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe5538-970f-4f91-aa01-e5703c266fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:10.346020 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe5538-970f-4f91-aa01-e5703c266fb8 admin admin] PATH_INFO: `/v3/projects/0c55dac7229641849299cf317ab34113/users/c6e47fe84ef643f4bcf68a17e90972e7/roles/54195e660fc349978b39d5794765a095` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:10.368745 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe5538-970f-4f91-aa01-e5703c266fb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c55dac7229641849299cf317ab34113, user_id=c6e47fe84ef643f4bcf68a17e90972e7, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:10.371654 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe5538-970f-4f91-aa01-e5703c266fb8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:10.405722 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 967/1919] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:10.406688 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ec283-ac66-46ea-99c7-ced87b60c7f5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:10.414134 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 953/1920] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:10 2022] PUT /identity/v3/projects/0c55dac7229641849299cf317ab34113/users/c6e47fe84ef643f4bcf68a17e90972e7/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:10.417711 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe5538-970f-4f91-aa01-e5703c266fb8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:10.421910 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:10.429688 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ec283-ac66-46ea-99c7-ced87b60c7f5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:10.467266 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:10.565746 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe5538-970f-4f91-aa01-e5703c266fb8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:10.569201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8fa286-1021-42c4-b32f-e426b50867f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:10.569532 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8fa286-1021-42c4-b32f-e426b50867f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:10.570368 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8fa286-1021-42c4-b32f-e426b50867f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:10.570625 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8fa286-1021-42c4-b32f-e426b50867f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:10.571849 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8fa286-1021-42c4-b32f-e426b50867f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:10.574121 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8fa286-1021-42c4-b32f-e426b50867f5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:10.590236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0d7d8f-5059-464f-af30-b11ddb848b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:10.590236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d7d8f-5059-464f-af30-b11ddb848b24 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:10.590951 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d7d8f-5059-464f-af30-b11ddb848b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:10.591238 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d7d8f-5059-464f-af30-b11ddb848b24 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:10.592174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0d7d8f-5059-464f-af30-b11ddb848b24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:05:10.592491 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0d7d8f-5059-464f-af30-b11ddb848b24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:10.594154 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0d7d8f-5059-464f-af30-b11ddb848b24 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:10.624245 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 968/1922] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:10 2022] POST /identity/v3/users => generated 301 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:10.625441 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c29649-400a-48ee-8352-1186b55fbc3a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:10.625757 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c29649-400a-48ee-8352-1186b55fbc3a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:10.626034 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c29649-400a-48ee-8352-1186b55fbc3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:10.627294 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 954/1922] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:05:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:05:10.628222 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0d7d8f-5059-464f-af30-b11ddb848b24 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:10.647472 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0d7d8f-5059-464f-af30-b11ddb848b24 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:10.676190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e724c5b8-8006-4ce9-82c1-76a24bce371b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:10.676881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724c5b8-8006-4ce9-82c1-76a24bce371b admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:10.677167 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724c5b8-8006-4ce9-82c1-76a24bce371b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:10.677446 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724c5b8-8006-4ce9-82c1-76a24bce371b admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:10.678184 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e724c5b8-8006-4ce9-82c1-76a24bce371b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:10.680146 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e724c5b8-8006-4ce9-82c1-76a24bce371b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:10.698762 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-41c29649-400a-48ee-8352-1186b55fbc3a None None] MFA Rules not processed for user `c6e47fe84ef643f4bcf68a17e90972e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:10.699985 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-41c29649-400a-48ee-8352-1186b55fbc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:10.737256 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 955/1923] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:10 2022] POST /identity/v3/projects => generated 401 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:10.738126 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e724c5b8-8006-4ce9-82c1-76a24bce371b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:10.801989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 969/1924] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:10.805343 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c29649-400a-48ee-8352-1186b55fbc3a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:10.820327 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c29649-400a-48ee-8352-1186b55fbc3a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:10.827911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac542d98-a401-457d-8c31-c2c8eb35d031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:10.828528 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac542d98-a401-457d-8c31-c2c8eb35d031 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:10.828880 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac542d98-a401-457d-8c31-c2c8eb35d031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:10.829228 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac542d98-a401-457d-8c31-c2c8eb35d031 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:10.831963 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac542d98-a401-457d-8c31-c2c8eb35d031 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:10.833882 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac542d98-a401-457d-8c31-c2c8eb35d031 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:10.835822 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e724c5b8-8006-4ce9-82c1-76a24bce371b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:10.844650 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca74fba-0914-49d8-9e37-a097766fc26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:10.845335 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca74fba-0914-49d8-9e37-a097766fc26a admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:10.845854 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca74fba-0914-49d8-9e37-a097766fc26a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:10.846193 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca74fba-0914-49d8-9e37-a097766fc26a admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:10.846931 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca74fba-0914-49d8-9e37-a097766fc26a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:10.848799 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca74fba-0914-49d8-9e37-a097766fc26a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:10.859917 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 956/1925] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:10.863153 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca74fba-0914-49d8-9e37-a097766fc26a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:10.873168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 970/1926] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:10 2022] POST /identity/v3/users => generated 301 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:10.873168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac542d98-a401-457d-8c31-c2c8eb35d031 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:10.873805 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca74fba-0914-49d8-9e37-a097766fc26a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:10.879096 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9ba9a-79c1-4596-b995-5a8d7d72d408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:10.879887 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9ba9a-79c1-4596-b995-5a8d7d72d408 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:10.880244 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9ba9a-79c1-4596-b995-5a8d7d72d408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:10.880606 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9ba9a-79c1-4596-b995-5a8d7d72d408 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:10.881397 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe9ba9a-79c1-4596-b995-5a8d7d72d408 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:10.886629 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac542d98-a401-457d-8c31-c2c8eb35d031 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:10.893418 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08d912-4da9-4a79-a9ea-86bcfb03e6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:10.894374 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08d912-4da9-4a79-a9ea-86bcfb03e6f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:10.894651 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08d912-4da9-4a79-a9ea-86bcfb03e6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:10.894931 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08d912-4da9-4a79-a9ea-86bcfb03e6f1 admin admin] PATH_INFO: `/v3/projects/1fa2bceb192e4817a5ba676203c47dbd/users/3ead0122048e4b6f99bc5542c5ec0502/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:10.902541 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe9ba9a-79c1-4596-b995-5a8d7d72d408 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:10.916527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f08d912-4da9-4a79-a9ea-86bcfb03e6f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fa2bceb192e4817a5ba676203c47dbd, user_id=3ead0122048e4b6f99bc5542c5ec0502, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:10.919314 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f08d912-4da9-4a79-a9ea-86bcfb03e6f1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:10.936705 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 971/1927] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:10 2022] PUT /identity/v3/projects/1fa2bceb192e4817a5ba676203c47dbd/users/3ead0122048e4b6f99bc5542c5ec0502/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:10.938004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08d912-4da9-4a79-a9ea-86bcfb03e6f1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:10.940582 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 957/1928] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:10 2022] POST /identity/v3/projects => generated 379 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:10.941542 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9ba9a-79c1-4596-b995-5a8d7d72d408 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:10.985984 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:11.006045 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9ba9a-79c1-4596-b995-5a8d7d72d408 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:11.007015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08d912-4da9-4a79-a9ea-86bcfb03e6f1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:11.011665 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9896655-0a67-429d-8d95-734d5430faa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:11.012340 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9896655-0a67-429d-8d95-734d5430faa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:11.012454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f02536-9415-40a7-b106-ad0cc5df19e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:11.012784 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9896655-0a67-429d-8d95-734d5430faa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:11.013193 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f02536-9415-40a7-b106-ad0cc5df19e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:11.013302 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9896655-0a67-429d-8d95-734d5430faa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:11.013471 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f02536-9415-40a7-b106-ad0cc5df19e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:11.013816 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f02536-9415-40a7-b106-ad0cc5df19e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:11.014066 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9896655-0a67-429d-8d95-734d5430faa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:11.014601 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f02536-9415-40a7-b106-ad0cc5df19e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:11.015725 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9896655-0a67-429d-8d95-734d5430faa7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:11.016259 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f02536-9415-40a7-b106-ad0cc5df19e8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:11.025880 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 958/1929] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:11.029080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 972/1930] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:11.029210 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9896655-0a67-429d-8d95-734d5430faa7 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:11.029322 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f02536-9415-40a7-b106-ad0cc5df19e8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:11.030573 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e0f02536-9415-40a7-b106-ad0cc5df19e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:11.032578 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:11.038170 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9896655-0a67-429d-8d95-734d5430faa7 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:11.043507 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f02536-9415-40a7-b106-ad0cc5df19e8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:11.043913 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e35689-0e0a-441a-acd4-7531b21d61ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:11.044581 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e35689-0e0a-441a-acd4-7531b21d61ef admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:11.044876 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e35689-0e0a-441a-acd4-7531b21d61ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:11.045161 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e35689-0e0a-441a-acd4-7531b21d61ef admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:11.045892 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e35689-0e0a-441a-acd4-7531b21d61ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:11.047867 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e35689-0e0a-441a-acd4-7531b21d61ef admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:11.048159 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191d4a9-8a13-4a8f-86e3-573f34c8f6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:11.049071 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191d4a9-8a13-4a8f-86e3-573f34c8f6c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:11.049355 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191d4a9-8a13-4a8f-86e3-573f34c8f6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:11.049679 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191d4a9-8a13-4a8f-86e3-573f34c8f6c1 admin admin] PATH_INFO: `/v3/users/86fd92f728f74c79a1a5b54d2aa8bf86` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:11.050878 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a191d4a9-8a13-4a8f-86e3-573f34c8f6c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86fd92f728f74c79a1a5b54d2aa8bf86)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:11.052788 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a191d4a9-8a13-4a8f-86e3-573f34c8f6c1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:11.071386 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 959/1931] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:11 2022] POST /identity/v3/users => generated 296 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:11.071386 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e35689-0e0a-441a-acd4-7531b21d61ef admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:11.079911 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e35689-0e0a-441a-acd4-7531b21d61ef admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:11.084814 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2c0e5d-6b3c-4069-ad78-a29f448caccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:11.085631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c0e5d-6b3c-4069-ad78-a29f448caccd admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:11.085971 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c0e5d-6b3c-4069-ad78-a29f448caccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:11.086327 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c0e5d-6b3c-4069-ad78-a29f448caccd admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:11.087189 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2c0e5d-6b3c-4069-ad78-a29f448caccd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:11.088889 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2c0e5d-6b3c-4069-ad78-a29f448caccd admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:11.099653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 960/1932] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:11.100519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2c0e5d-6b3c-4069-ad78-a29f448caccd admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:11.102944 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a191d4a9-8a13-4a8f-86e3-573f34c8f6c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86fd92f728f74c79a1a5b54d2aa8bf86'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:11.103685 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a191d4a9-8a13-4a8f-86e3-573f34c8f6c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86fd92f728f74c79a1a5b54d2aa8bf86'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:11.104304 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a191d4a9-8a13-4a8f-86e3-573f34c8f6c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86fd92f728f74c79a1a5b54d2aa8bf86'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:11.110480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2c0e5d-6b3c-4069-ad78-a29f448caccd admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:11.111348 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a191d4a9-8a13-4a8f-86e3-573f34c8f6c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86fd92f728f74c79a1a5b54d2aa8bf86'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:11.115167 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd2f961-a8bf-4cd0-9b85-c3d4f7a3c035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:11.116135 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2f961-a8bf-4cd0-9b85-c3d4f7a3c035 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:11.116951 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2f961-a8bf-4cd0-9b85-c3d4f7a3c035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:11.117278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2f961-a8bf-4cd0-9b85-c3d4f7a3c035 admin admin] PATH_INFO: `/v3/projects/1fa2bceb192e4817a5ba676203c47dbd/users/3ead0122048e4b6f99bc5542c5ec0502/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:11.119529 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd2f961-a8bf-4cd0-9b85-c3d4f7a3c035 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fa2bceb192e4817a5ba676203c47dbd, user_id=3ead0122048e4b6f99bc5542c5ec0502, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:11.119817 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a191d4a9-8a13-4a8f-86e3-573f34c8f6c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86fd92f728f74c79a1a5b54d2aa8bf86'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:11.122406 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd2f961-a8bf-4cd0-9b85-c3d4f7a3c035 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:11.133852 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 961/1933] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:11 2022] PUT /identity/v3/projects/1fa2bceb192e4817a5ba676203c47dbd/users/3ead0122048e4b6f99bc5542c5ec0502/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:11.134438 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd2f961-a8bf-4cd0-9b85-c3d4f7a3c035 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:11.135967 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9cd2f961-a8bf-4cd0-9b85-c3d4f7a3c035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:11.138569 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 973/1934] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:11 2022] DELETE /identity/v3/users/86fd92f728f74c79a1a5b54d2aa8bf86 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:11.139339 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191d4a9-8a13-4a8f-86e3-573f34c8f6c1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:11.140800 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a191d4a9-8a13-4a8f-86e3-573f34c8f6c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:11.202311 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd2f961-a8bf-4cd0-9b85-c3d4f7a3c035 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:11.204814 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191d4a9-8a13-4a8f-86e3-573f34c8f6c1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:11.207125 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24990e59-412d-4fe8-80c0-014070275484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:11.207802 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24990e59-412d-4fe8-80c0-014070275484 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:11.208086 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24990e59-412d-4fe8-80c0-014070275484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:11.208367 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24990e59-412d-4fe8-80c0-014070275484 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:11.209108 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24990e59-412d-4fe8-80c0-014070275484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:11.209492 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86e5a9-f434-4718-a107-6716afbf5284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:11.210431 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86e5a9-f434-4718-a107-6716afbf5284 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:11.210714 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86e5a9-f434-4718-a107-6716afbf5284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:11.210815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24990e59-412d-4fe8-80c0-014070275484 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:11.210988 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86e5a9-f434-4718-a107-6716afbf5284 admin admin] PATH_INFO: `/v3/projects/d708173f297c44d08f289fbda4c0b051/users/0160daa137e640faac805671e0fafe38/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:11.220923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 962/1935] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:11.222351 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c3f6d-23d9-4e07-96f6-3a38a229819c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:11.222649 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c3f6d-23d9-4e07-96f6-3a38a229819c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:11.222936 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c3f6d-23d9-4e07-96f6-3a38a229819c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:11.228487 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b86e5a9-f434-4718-a107-6716afbf5284 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d708173f297c44d08f289fbda4c0b051, user_id=0160daa137e640faac805671e0fafe38, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:11.231259 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b86e5a9-f434-4718-a107-6716afbf5284 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:11.241304 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5e4c3f6d-23d9-4e07-96f6-3a38a229819c None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:11.243134 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5e4c3f6d-23d9-4e07-96f6-3a38a229819c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:11.251157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 963/1936] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:11.252521 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a629a4-64ba-4501-b272-e41a95c5154b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:11.252817 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a629a4-64ba-4501-b272-e41a95c5154b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:11.253098 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a629a4-64ba-4501-b272-e41a95c5154b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:11.253815 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 974/1937] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:11 2022] PUT /identity/v3/projects/d708173f297c44d08f289fbda4c0b051/users/0160daa137e640faac805671e0fafe38/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:11.258585 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86e5a9-f434-4718-a107-6716afbf5284 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:11.283301 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d7a629a4-64ba-4501-b272-e41a95c5154b None None] MFA Rules not processed for user `3ead0122048e4b6f99bc5542c5ec0502`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:11.284510 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d7a629a4-64ba-4501-b272-e41a95c5154b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:11.294722 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:11.325547 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86e5a9-f434-4718-a107-6716afbf5284 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:11.330193 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae0ba1-e6da-42f6-b639-9bfc3aa56e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:11.331075 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae0ba1-e6da-42f6-b639-9bfc3aa56e2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:11.331358 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae0ba1-e6da-42f6-b639-9bfc3aa56e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:11.331633 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae0ba1-e6da-42f6-b639-9bfc3aa56e2e admin admin] PATH_INFO: `/v3/projects/48cc9814518d414aa75786a356915354/users/dd2d054d4a76428385df2a856f3dfe9e/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:11.348434 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae0ba1-e6da-42f6-b639-9bfc3aa56e2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=48cc9814518d414aa75786a356915354, user_id=dd2d054d4a76428385df2a856f3dfe9e, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:11.351232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae0ba1-e6da-42f6-b639-9bfc3aa56e2e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:11.355852 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 964/1938] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:11 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:11.356628 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a629a4-64ba-4501-b272-e41a95c5154b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:11.357980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d7a629a4-64ba-4501-b272-e41a95c5154b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:11.369162 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 975/1939] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:11 2022] PUT /identity/v3/projects/48cc9814518d414aa75786a356915354/users/dd2d054d4a76428385df2a856f3dfe9e/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:11.373286 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc04ac-9d23-4672-bc39-051ec06b6d4b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:11.373603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc04ac-9d23-4672-bc39-051ec06b6d4b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:11.373884 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc04ac-9d23-4672-bc39-051ec06b6d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:11.398682 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:11.400706 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e6bc04ac-9d23-4672-bc39-051ec06b6d4b None None] MFA Rules not processed for user `0160daa137e640faac805671e0fafe38`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:11.402068 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e6bc04ac-9d23-4672-bc39-051ec06b6d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:11.419846 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a629a4-64ba-4501-b272-e41a95c5154b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:11.465348 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 976/1940] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:11 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:11.466558 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc04ac-9d23-4672-bc39-051ec06b6d4b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:11.468000 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e6bc04ac-9d23-4672-bc39-051ec06b6d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:11.469590 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440342b-3eda-475b-931d-80d208b7105f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:11.470251 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440342b-3eda-475b-931d-80d208b7105f service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:11.470541 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440342b-3eda-475b-931d-80d208b7105f service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:11.470822 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440342b-3eda-475b-931d-80d208b7105f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:11.527222 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc04ac-9d23-4672-bc39-051ec06b6d4b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:11.530935 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9765924-de15-43c7-826d-7251f29e50b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:11.531612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9765924-de15-43c7-826d-7251f29e50b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:11.531896 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9765924-de15-43c7-826d-7251f29e50b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:11.532171 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9765924-de15-43c7-826d-7251f29e50b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:11.532375 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440342b-3eda-475b-931d-80d208b7105f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:11.533849 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440342b-3eda-475b-931d-80d208b7105f service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:11.543552 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 965/1941] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:11.544397 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440342b-3eda-475b-931d-80d208b7105f service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:11.553965 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440342b-3eda-475b-931d-80d208b7105f service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:11.559359 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2acd82-7fa0-4c90-ae8f-e316ae2fd021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:11.560051 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2acd82-7fa0-4c90-ae8f-e316ae2fd021 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:11.560339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2acd82-7fa0-4c90-ae8f-e316ae2fd021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:11.560628 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2acd82-7fa0-4c90-ae8f-e316ae2fd021 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:11.561377 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2acd82-7fa0-4c90-ae8f-e316ae2fd021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:11.563160 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2acd82-7fa0-4c90-ae8f-e316ae2fd021 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:11.574297 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 966/1942] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:11.574924 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2acd82-7fa0-4c90-ae8f-e316ae2fd021 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:11.586087 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2acd82-7fa0-4c90-ae8f-e316ae2fd021 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:11.591111 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298fbb8a-8ace-457e-9440-fb6dcc942d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:11.591786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fbb8a-8ace-457e-9440-fb6dcc942d78 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:11.592075 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fbb8a-8ace-457e-9440-fb6dcc942d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:11.592355 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fbb8a-8ace-457e-9440-fb6dcc942d78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:11.593526 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298fbb8a-8ace-457e-9440-fb6dcc942d78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:11.595803 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298fbb8a-8ace-457e-9440-fb6dcc942d78 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:11.605959 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:11.610431 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9765924-de15-43c7-826d-7251f29e50b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:11.613686 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9765924-de15-43c7-826d-7251f29e50b8 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:11.625820 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 967/1943] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:11 2022] POST /identity/v3/projects => generated 399 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:11.626905 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298fbb8a-8ace-457e-9440-fb6dcc942d78 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:11.639727 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 977/1944] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:11 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:11.640979 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9765924-de15-43c7-826d-7251f29e50b8 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:11.658517 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9765924-de15-43c7-826d-7251f29e50b8 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:11.712388 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298fbb8a-8ace-457e-9440-fb6dcc942d78 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:11.719904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef86dffe-8c35-4d09-942e-2b5eb3389b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:11.720832 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef86dffe-8c35-4d09-942e-2b5eb3389b05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:11.721123 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef86dffe-8c35-4d09-942e-2b5eb3389b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:11.723391 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef86dffe-8c35-4d09-942e-2b5eb3389b05 admin admin] PATH_INFO: `/v3/projects/48cc9814518d414aa75786a356915354/users/dd2d054d4a76428385df2a856f3dfe9e/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:11.729793 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef86dffe-8c35-4d09-942e-2b5eb3389b05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48cc9814518d414aa75786a356915354, user_id=dd2d054d4a76428385df2a856f3dfe9e, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:11.733010 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef86dffe-8c35-4d09-942e-2b5eb3389b05 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:11.753658 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 968/1945] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:11 2022] PUT /identity/v3/projects/48cc9814518d414aa75786a356915354/users/dd2d054d4a76428385df2a856f3dfe9e/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:11.762273 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21f4d6-eba3-402e-ba87-5b42fc761a8f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:11.762579 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21f4d6-eba3-402e-ba87-5b42fc761a8f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:11.762859 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21f4d6-eba3-402e-ba87-5b42fc761a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:11.769852 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:11.772498 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098a813-bfde-4a9e-a6bf-8814df537e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:11.775590 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098a813-bfde-4a9e-a6bf-8814df537e49 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:11.776069 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098a813-bfde-4a9e-a6bf-8814df537e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:11.776912 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098a813-bfde-4a9e-a6bf-8814df537e49 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:11.783273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3098a813-bfde-4a9e-a6bf-8814df537e49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:11.786757 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3098a813-bfde-4a9e-a6bf-8814df537e49 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:11.818688 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1c21f4d6-eba3-402e-ba87-5b42fc761a8f None None] MFA Rules not processed for user `dd2d054d4a76428385df2a856f3dfe9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:11.820292 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1c21f4d6-eba3-402e-ba87-5b42fc761a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:11.845369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 978/1946] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:11 2022] POST /identity/v3/users => generated 306 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:11.845369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098a813-bfde-4a9e-a6bf-8814df537e49 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:11.845369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3098a813-bfde-4a9e-a6bf-8814df537e49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:11.858853 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098a813-bfde-4a9e-a6bf-8814df537e49 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:11.940976 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 969/1947] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:11 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:05:11.942734 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21f4d6-eba3-402e-ba87-5b42fc761a8f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:11.955024 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef8f34-2ef1-4dac-8bdd-037a07a0a98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:11.956113 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef8f34-2ef1-4dac-8bdd-037a07a0a98a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:11.956529 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef8f34-2ef1-4dac-8bdd-037a07a0a98a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:11.956814 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef8f34-2ef1-4dac-8bdd-037a07a0a98a admin admin] PATH_INFO: `/v3/users/69bf76277d6e4577a826e7ec3dcfe708` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:11.960918 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef8f34-2ef1-4dac-8bdd-037a07a0a98a admin admin] RBAC: Authorizing `identity:delete_user(user_id=69bf76277d6e4577a826e7ec3dcfe708)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:11.963570 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef8f34-2ef1-4dac-8bdd-037a07a0a98a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:12.033692 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-10ef8f34-2ef1-4dac-8bdd-037a07a0a98a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69bf76277d6e4577a826e7ec3dcfe708'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:12.034373 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-10ef8f34-2ef1-4dac-8bdd-037a07a0a98a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69bf76277d6e4577a826e7ec3dcfe708'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:12.035299 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-10ef8f34-2ef1-4dac-8bdd-037a07a0a98a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69bf76277d6e4577a826e7ec3dcfe708'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:12.044243 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-10ef8f34-2ef1-4dac-8bdd-037a07a0a98a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69bf76277d6e4577a826e7ec3dcfe708'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:12.058379 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21f4d6-eba3-402e-ba87-5b42fc761a8f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:12.063368 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-10ef8f34-2ef1-4dac-8bdd-037a07a0a98a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69bf76277d6e4577a826e7ec3dcfe708'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:12.064253 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af36f1d-9c6e-4e81-827a-ca7cea1630d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:12.064986 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af36f1d-9c6e-4e81-827a-ca7cea1630d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:12.065277 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af36f1d-9c6e-4e81-827a-ca7cea1630d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:12.065605 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af36f1d-9c6e-4e81-827a-ca7cea1630d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:12.066542 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af36f1d-9c6e-4e81-827a-ca7cea1630d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:12.068145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af36f1d-9c6e-4e81-827a-ca7cea1630d8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:12.083251 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 979/1948] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:11 2022] DELETE /identity/v3/users/69bf76277d6e4577a826e7ec3dcfe708 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:12.084091 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef8f34-2ef1-4dac-8bdd-037a07a0a98a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:12.086174 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-10ef8f34-2ef1-4dac-8bdd-037a07a0a98a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:12.088931 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 970/1949] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:12.089857 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af36f1d-9c6e-4e81-827a-ca7cea1630d8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:12.091452 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0af36f1d-9c6e-4e81-827a-ca7cea1630d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:12.093398 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:12.106755 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af36f1d-9c6e-4e81-827a-ca7cea1630d8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:12.148699 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef8f34-2ef1-4dac-8bdd-037a07a0a98a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:12.153444 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f8e15e-5d15-4886-8bcd-203ca1010276 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:12.154442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8e15e-5d15-4886-8bcd-203ca1010276 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:12.154736 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8e15e-5d15-4886-8bcd-203ca1010276 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:12.155021 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8e15e-5d15-4886-8bcd-203ca1010276 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:12.156807 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-72f8e15e-5d15-4886-8bcd-203ca1010276 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:12.169876 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd635a5b-fdc0-439e-8b20-7c5ec7f80895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:12.170753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd635a5b-fdc0-439e-8b20-7c5ec7f80895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:12.171042 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd635a5b-fdc0-439e-8b20-7c5ec7f80895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:12.171324 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd635a5b-fdc0-439e-8b20-7c5ec7f80895 admin admin] PATH_INFO: `/v3/projects/c03c670b45414ce09aa629295338f815` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:12.172522 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd635a5b-fdc0-439e-8b20-7c5ec7f80895 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c03c670b45414ce09aa629295338f815)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:12.174490 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd635a5b-fdc0-439e-8b20-7c5ec7f80895 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:12.208908 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:12.228924 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f8e15e-5d15-4886-8bcd-203ca1010276 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:12.231266 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f8e15e-5d15-4886-8bcd-203ca1010276 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:12.237644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-dd635a5b-fdc0-439e-8b20-7c5ec7f80895 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c03c670b45414ce09aa629295338f815', 'request_id': 'req-dd635a5b-fdc0-439e-8b20-7c5ec7f80895'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:12.244274 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 980/1950] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:12 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:12.245633 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63fb9ea-5fdd-4053-baf8-d4f09629cd3b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:12.245909 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63fb9ea-5fdd-4053-baf8-d4f09629cd3b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:12.246191 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63fb9ea-5fdd-4053-baf8-d4f09629cd3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:12.250551 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-dd635a5b-fdc0-439e-8b20-7c5ec7f80895 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c03c670b45414ce09aa629295338f815', 'request_id': 'req-dd635a5b-fdc0-439e-8b20-7c5ec7f80895'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:12.262423 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-dd635a5b-fdc0-439e-8b20-7c5ec7f80895 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c03c670b45414ce09aa629295338f815', 'request_id': 'req-dd635a5b-fdc0-439e-8b20-7c5ec7f80895'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:12.268192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e63fb9ea-5fdd-4053-baf8-d4f09629cd3b None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:12.269408 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e63fb9ea-5fdd-4053-baf8-d4f09629cd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:12.275366 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-dd635a5b-fdc0-439e-8b20-7c5ec7f80895 admin admin] The token cache is being invalidate because project c03c670b45414ce09aa629295338f815 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:12.275730 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-dd635a5b-fdc0-439e-8b20-7c5ec7f80895 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:12.276542 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-dd635a5b-fdc0-439e-8b20-7c5ec7f80895 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:12.278274 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 971/1951] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:12 2022] DELETE /identity/v3/projects/c03c670b45414ce09aa629295338f815 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:12.279102 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd635a5b-fdc0-439e-8b20-7c5ec7f80895 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:12.280968 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-dd635a5b-fdc0-439e-8b20-7c5ec7f80895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:12.361331 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd635a5b-fdc0-439e-8b20-7c5ec7f80895 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:12.366017 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82872a8e-9a9d-496e-a59b-171ce830df15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:12.366955 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82872a8e-9a9d-496e-a59b-171ce830df15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:12.367249 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82872a8e-9a9d-496e-a59b-171ce830df15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:12.367533 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82872a8e-9a9d-496e-a59b-171ce830df15 admin admin] PATH_INFO: `/v3/projects/5e8155b725ec4a9e941adbe6390f306f/users/cdd0b902a18c45609044c385fa364f77/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:12.369162 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 981/1952] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:12.370145 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63fb9ea-5fdd-4053-baf8-d4f09629cd3b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:12.378089 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e63fb9ea-5fdd-4053-baf8-d4f09629cd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:12.388074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82872a8e-9a9d-496e-a59b-171ce830df15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e8155b725ec4a9e941adbe6390f306f, user_id=cdd0b902a18c45609044c385fa364f77, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:12.390784 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82872a8e-9a9d-496e-a59b-171ce830df15 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:12.390985 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63fb9ea-5fdd-4053-baf8-d4f09629cd3b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:12.396678 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98e0c0-c221-4a72-8c7f-10b0fa215a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:12.398412 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98e0c0-c221-4a72-8c7f-10b0fa215a5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:12.398739 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98e0c0-c221-4a72-8c7f-10b0fa215a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:12.399288 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98e0c0-c221-4a72-8c7f-10b0fa215a5a admin admin] PATH_INFO: `/v3/users/d094fb4900ca4da4ade96399402dee4b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:12.400641 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c98e0c0-c221-4a72-8c7f-10b0fa215a5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d094fb4900ca4da4ade96399402dee4b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:12.402940 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c98e0c0-c221-4a72-8c7f-10b0fa215a5a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:12.407931 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 972/1953] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:12 2022] PUT /identity/v3/projects/5e8155b725ec4a9e941adbe6390f306f/users/cdd0b902a18c45609044c385fa364f77/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:12.408552 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82872a8e-9a9d-496e-a59b-171ce830df15 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:12.411114 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-82872a8e-9a9d-496e-a59b-171ce830df15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:12.464195 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2c98e0c0-c221-4a72-8c7f-10b0fa215a5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd094fb4900ca4da4ade96399402dee4b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:12.464873 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2c98e0c0-c221-4a72-8c7f-10b0fa215a5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd094fb4900ca4da4ade96399402dee4b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:12.469215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2c98e0c0-c221-4a72-8c7f-10b0fa215a5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd094fb4900ca4da4ade96399402dee4b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:12.469952 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:12.476968 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2c98e0c0-c221-4a72-8c7f-10b0fa215a5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd094fb4900ca4da4ade96399402dee4b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:12.485367 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82872a8e-9a9d-496e-a59b-171ce830df15 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:12.489283 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b56eb43-1614-489a-8995-8e7dac173be8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:12.490028 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56eb43-1614-489a-8995-8e7dac173be8 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:12.490263 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56eb43-1614-489a-8995-8e7dac173be8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:12.490538 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56eb43-1614-489a-8995-8e7dac173be8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:12.490646 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2c98e0c0-c221-4a72-8c7f-10b0fa215a5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd094fb4900ca4da4ade96399402dee4b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:12.492656 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1b56eb43-1614-489a-8995-8e7dac173be8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:12.521927 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 982/1954] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:12 2022] DELETE /identity/v3/users/d094fb4900ca4da4ade96399402dee4b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:12.524014 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98e0c0-c221-4a72-8c7f-10b0fa215a5a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:12.525797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2c98e0c0-c221-4a72-8c7f-10b0fa215a5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:12.547141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:12.564707 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b56eb43-1614-489a-8995-8e7dac173be8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:12.566306 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b56eb43-1614-489a-8995-8e7dac173be8 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:12.577677 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 973/1955] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:12.639351 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98e0c0-c221-4a72-8c7f-10b0fa215a5a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:12.646153 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0539e-16c3-4c8b-9fef-266abcbd7dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:12.646903 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0539e-16c3-4c8b-9fef-266abcbd7dca admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:12.647501 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0539e-16c3-4c8b-9fef-266abcbd7dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:12.647612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0539e-16c3-4c8b-9fef-266abcbd7dca admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:12.649286 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c0539e-16c3-4c8b-9fef-266abcbd7dca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:12.651501 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c0539e-16c3-4c8b-9fef-266abcbd7dca admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:12.664702 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 983/1956] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:12.668786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b56eb43-1614-489a-8995-8e7dac173be8 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:12.679265 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b56eb43-1614-489a-8995-8e7dac173be8 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:12.684887 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c101c-24a0-476e-b3b0-4be1c300bb33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:12.685954 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c101c-24a0-476e-b3b0-4be1c300bb33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:12.686246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c101c-24a0-476e-b3b0-4be1c300bb33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:12.686531 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c101c-24a0-476e-b3b0-4be1c300bb33 admin admin] PATH_INFO: `/v3/projects/5e8155b725ec4a9e941adbe6390f306f/users/cdd0b902a18c45609044c385fa364f77/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:12.688545 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c101c-24a0-476e-b3b0-4be1c300bb33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e8155b725ec4a9e941adbe6390f306f, user_id=cdd0b902a18c45609044c385fa364f77, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:12.691387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c101c-24a0-476e-b3b0-4be1c300bb33 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:12.697566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0539e-16c3-4c8b-9fef-266abcbd7dca admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:12.700975 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-61c0539e-16c3-4c8b-9fef-266abcbd7dca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:12.705464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 974/1957] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:12 2022] PUT /identity/v3/projects/5e8155b725ec4a9e941adbe6390f306f/users/cdd0b902a18c45609044c385fa364f77/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:12.717362 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0539e-16c3-4c8b-9fef-266abcbd7dca admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:12.717613 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb156c65-8399-4588-9d3b-543f36cf6f45 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:12.717786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb156c65-8399-4588-9d3b-543f36cf6f45 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:12.718073 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb156c65-8399-4588-9d3b-543f36cf6f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:12.751009 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-bb156c65-8399-4588-9d3b-543f36cf6f45 None None] MFA Rules not processed for user `cdd0b902a18c45609044c385fa364f77`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:12.752453 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bb156c65-8399-4588-9d3b-543f36cf6f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:12.787722 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8c1cf-7122-4c9f-83a0-8b8e6c114db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:12.789086 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8c1cf-7122-4c9f-83a0-8b8e6c114db0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:12.789086 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8c1cf-7122-4c9f-83a0-8b8e6c114db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:12.789529 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8c1cf-7122-4c9f-83a0-8b8e6c114db0 admin admin] PATH_INFO: `/v3/users/c6e47fe84ef643f4bcf68a17e90972e7` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:12.791043 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c8c1cf-7122-4c9f-83a0-8b8e6c114db0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6e47fe84ef643f4bcf68a17e90972e7)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:12.794041 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c8c1cf-7122-4c9f-83a0-8b8e6c114db0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:12.874083 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e2c8c1cf-7122-4c9f-83a0-8b8e6c114db0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6e47fe84ef643f4bcf68a17e90972e7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:12.874361 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 975/1958] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:12 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:12.875418 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e2c8c1cf-7122-4c9f-83a0-8b8e6c114db0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6e47fe84ef643f4bcf68a17e90972e7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:12.876089 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e2c8c1cf-7122-4c9f-83a0-8b8e6c114db0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6e47fe84ef643f4bcf68a17e90972e7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:12.889484 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb156c65-8399-4588-9d3b-543f36cf6f45 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:12.890161 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e2c8c1cf-7122-4c9f-83a0-8b8e6c114db0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6e47fe84ef643f4bcf68a17e90972e7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:12.891425 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bb156c65-8399-4588-9d3b-543f36cf6f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:12.904683 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e2c8c1cf-7122-4c9f-83a0-8b8e6c114db0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6e47fe84ef643f4bcf68a17e90972e7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:12.907712 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:12.945904 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 984/1959] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:12 2022] DELETE /identity/v3/users/c6e47fe84ef643f4bcf68a17e90972e7 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:05:12.950464 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07211f2-17db-4c24-b92b-07fc155e6fd3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:12.951037 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07211f2-17db-4c24-b92b-07fc155e6fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:12.952014 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07211f2-17db-4c24-b92b-07fc155e6fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:12.979374 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a07211f2-17db-4c24-b92b-07fc155e6fd3 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:12.980671 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a07211f2-17db-4c24-b92b-07fc155e6fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:12.983726 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb156c65-8399-4588-9d3b-543f36cf6f45 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:13.052523 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bfe07b-d471-4540-8d1f-d001139ba8ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:13.053665 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 985/1960] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:13.053977 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bfe07b-d471-4540-8d1f-d001139ba8ed service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:13.054809 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bfe07b-d471-4540-8d1f-d001139ba8ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:13.056578 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07211f2-17db-4c24-b92b-07fc155e6fd3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:13.056686 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bfe07b-d471-4540-8d1f-d001139ba8ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:13.057592 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a07211f2-17db-4c24-b92b-07fc155e6fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:13.067832 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d4bfe07b-d471-4540-8d1f-d001139ba8ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:13.069778 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07211f2-17db-4c24-b92b-07fc155e6fd3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:13.076550 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e84701c-c894-478a-bcca-32d7b4b6545d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:13.077526 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84701c-c894-478a-bcca-32d7b4b6545d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:13.077916 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84701c-c894-478a-bcca-32d7b4b6545d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:13.078203 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84701c-c894-478a-bcca-32d7b4b6545d admin admin] PATH_INFO: `/v3/projects/a6e9793ba4c44cebb349be26f7baba63` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:13.079525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e84701c-c894-478a-bcca-32d7b4b6545d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6e9793ba4c44cebb349be26f7baba63)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:13.081778 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e84701c-c894-478a-bcca-32d7b4b6545d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:13.121601 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:13.148761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4e84701c-c894-478a-bcca-32d7b4b6545d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6e9793ba4c44cebb349be26f7baba63', 'request_id': 'req-4e84701c-c894-478a-bcca-32d7b4b6545d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:13.156921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4e84701c-c894-478a-bcca-32d7b4b6545d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6e9793ba4c44cebb349be26f7baba63', 'request_id': 'req-4e84701c-c894-478a-bcca-32d7b4b6545d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:13.164710 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4e84701c-c894-478a-bcca-32d7b4b6545d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6e9793ba4c44cebb349be26f7baba63', 'request_id': 'req-4e84701c-c894-478a-bcca-32d7b4b6545d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:13.168673 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bfe07b-d471-4540-8d1f-d001139ba8ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:13.170313 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bfe07b-d471-4540-8d1f-d001139ba8ed service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:13.177273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4e84701c-c894-478a-bcca-32d7b4b6545d admin admin] The token cache is being invalidate because project a6e9793ba4c44cebb349be26f7baba63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:13.177600 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4e84701c-c894-478a-bcca-32d7b4b6545d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:13.178277 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4e84701c-c894-478a-bcca-32d7b4b6545d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:13.180057 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 986/1961] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:13 2022] DELETE /identity/v3/projects/a6e9793ba4c44cebb349be26f7baba63 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:13.181378 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e84701c-c894-478a-bcca-32d7b4b6545d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:13.182691 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4e84701c-c894-478a-bcca-32d7b4b6545d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:13.185195 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 976/1962] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:12 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 16:05:13.186117 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bfe07b-d471-4540-8d1f-d001139ba8ed service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:13.187586 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d4bfe07b-d471-4540-8d1f-d001139ba8ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:13.270493 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e84701c-c894-478a-bcca-32d7b4b6545d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:13.277448 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b683fc-56c6-4936-85d2-ae962e028ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:13.278462 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b683fc-56c6-4936-85d2-ae962e028ef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:13.278947 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b683fc-56c6-4936-85d2-ae962e028ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:13.279812 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b683fc-56c6-4936-85d2-ae962e028ef1 admin admin] PATH_INFO: `/v3/projects/29c95ae60ca14ab897570df35995ab15` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:13.281341 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bfe07b-d471-4540-8d1f-d001139ba8ed service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:13.281510 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b683fc-56c6-4936-85d2-ae962e028ef1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29c95ae60ca14ab897570df35995ab15)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:13.283799 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b683fc-56c6-4936-85d2-ae962e028ef1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:13.294594 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:13.354051 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f49b3-15aa-4334-a62d-e44183dab470 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:13.354321 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f49b3-15aa-4334-a62d-e44183dab470 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:13.354694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f49b3-15aa-4334-a62d-e44183dab470 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:13.354975 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f49b3-15aa-4334-a62d-e44183dab470 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:13.356817 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-831f49b3-15aa-4334-a62d-e44183dab470 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:13.361039 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-77b683fc-56c6-4936-85d2-ae962e028ef1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29c95ae60ca14ab897570df35995ab15', 'request_id': 'req-77b683fc-56c6-4936-85d2-ae962e028ef1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:13.371048 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831f49b3-15aa-4334-a62d-e44183dab470 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:13.371274 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-77b683fc-56c6-4936-85d2-ae962e028ef1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29c95ae60ca14ab897570df35995ab15', 'request_id': 'req-77b683fc-56c6-4936-85d2-ae962e028ef1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:13.373879 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831f49b3-15aa-4334-a62d-e44183dab470 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:13.385602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 977/1963] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:13.386237 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f49b3-15aa-4334-a62d-e44183dab470 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:13.387529 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-831f49b3-15aa-4334-a62d-e44183dab470 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:13.394334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-77b683fc-56c6-4936-85d2-ae962e028ef1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29c95ae60ca14ab897570df35995ab15', 'request_id': 'req-77b683fc-56c6-4936-85d2-ae962e028ef1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:13.400540 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f49b3-15aa-4334-a62d-e44183dab470 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:13.408580 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-77b683fc-56c6-4936-85d2-ae962e028ef1 admin admin] The token cache is being invalidate because project 29c95ae60ca14ab897570df35995ab15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:13.408906 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-77b683fc-56c6-4936-85d2-ae962e028ef1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:13.409614 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-77b683fc-56c6-4936-85d2-ae962e028ef1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:13.411182 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 987/1964] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:13 2022] DELETE /identity/v3/projects/29c95ae60ca14ab897570df35995ab15 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:13.411951 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b683fc-56c6-4936-85d2-ae962e028ef1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:13.413331 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-77b683fc-56c6-4936-85d2-ae962e028ef1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:13.478139 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b683fc-56c6-4936-85d2-ae962e028ef1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:13.479128 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db3046-830a-45f3-aedc-bbfab98796f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:13.480143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db3046-830a-45f3-aedc-bbfab98796f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:13.480426 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db3046-830a-45f3-aedc-bbfab98796f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:13.481033 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db3046-830a-45f3-aedc-bbfab98796f4 admin admin] PATH_INFO: `/v3/users/69f3459ed7344c2dbb465127dd662271` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:13.482473 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b67d75-40f2-4521-bc32-0cecae67e268 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:13.482787 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95db3046-830a-45f3-aedc-bbfab98796f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69f3459ed7344c2dbb465127dd662271)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:13.483255 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b67d75-40f2-4521-bc32-0cecae67e268 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:13.483604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b67d75-40f2-4521-bc32-0cecae67e268 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:13.483948 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b67d75-40f2-4521-bc32-0cecae67e268 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:13.485109 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95db3046-830a-45f3-aedc-bbfab98796f4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:13.485968 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-04b67d75-40f2-4521-bc32-0cecae67e268 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:13.526455 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:13.555996 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-95db3046-830a-45f3-aedc-bbfab98796f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69f3459ed7344c2dbb465127dd662271'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:13.557006 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-95db3046-830a-45f3-aedc-bbfab98796f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69f3459ed7344c2dbb465127dd662271'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:13.557738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-95db3046-830a-45f3-aedc-bbfab98796f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69f3459ed7344c2dbb465127dd662271'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:13.565747 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-95db3046-830a-45f3-aedc-bbfab98796f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69f3459ed7344c2dbb465127dd662271'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:13.572215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b67d75-40f2-4521-bc32-0cecae67e268 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:13.573782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b67d75-40f2-4521-bc32-0cecae67e268 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:13.574401 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-95db3046-830a-45f3-aedc-bbfab98796f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69f3459ed7344c2dbb465127dd662271'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:13.584399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 988/1965] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:13 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:13.585262 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b67d75-40f2-4521-bc32-0cecae67e268 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:13.586752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-04b67d75-40f2-4521-bc32-0cecae67e268 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:13.595891 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 978/1966] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:13 2022] DELETE /identity/v3/users/69f3459ed7344c2dbb465127dd662271 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:13.602188 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b67d75-40f2-4521-bc32-0cecae67e268 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:13.711903 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:13.730200 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df3be59-c5af-417b-a5c1-c721f86afd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:13.731711 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3be59-c5af-417b-a5c1-c721f86afd39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:13.732775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3be59-c5af-417b-a5c1-c721f86afd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:13.734138 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3be59-c5af-417b-a5c1-c721f86afd39 admin admin] PATH_INFO: `/v3/users/3ead0122048e4b6f99bc5542c5ec0502` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:13.736459 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df3be59-c5af-417b-a5c1-c721f86afd39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ead0122048e4b6f99bc5542c5ec0502)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:13.741162 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df3be59-c5af-417b-a5c1-c721f86afd39 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:13.783542 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db3046-830a-45f3-aedc-bbfab98796f4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:13.785179 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-95db3046-830a-45f3-aedc-bbfab98796f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:13.807091 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db3046-830a-45f3-aedc-bbfab98796f4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:13.815392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b1a35b-bdb7-4014-80ca-d00af8fbbafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:13.816307 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1a35b-bdb7-4014-80ca-d00af8fbbafd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:13.816603 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1a35b-bdb7-4014-80ca-d00af8fbbafd admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:13.816881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1a35b-bdb7-4014-80ca-d00af8fbbafd admin admin] PATH_INFO: `/v3/projects/0c55dac7229641849299cf317ab34113` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:13.818162 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b1a35b-bdb7-4014-80ca-d00af8fbbafd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c55dac7229641849299cf317ab34113)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:13.820370 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b1a35b-bdb7-4014-80ca-d00af8fbbafd admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:13.847143 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4df3be59-c5af-417b-a5c1-c721f86afd39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ead0122048e4b6f99bc5542c5ec0502'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:13.849813 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4df3be59-c5af-417b-a5c1-c721f86afd39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ead0122048e4b6f99bc5542c5ec0502'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:13.850686 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4df3be59-c5af-417b-a5c1-c721f86afd39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ead0122048e4b6f99bc5542c5ec0502'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:13.869828 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4df3be59-c5af-417b-a5c1-c721f86afd39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ead0122048e4b6f99bc5542c5ec0502'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:13.885165 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4df3be59-c5af-417b-a5c1-c721f86afd39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ead0122048e4b6f99bc5542c5ec0502'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:13.887130 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-45b1a35b-bdb7-4014-80ca-d00af8fbbafd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c55dac7229641849299cf317ab34113', 'request_id': 'req-45b1a35b-bdb7-4014-80ca-d00af8fbbafd'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:13.903944 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-45b1a35b-bdb7-4014-80ca-d00af8fbbafd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c55dac7229641849299cf317ab34113', 'request_id': 'req-45b1a35b-bdb7-4014-80ca-d00af8fbbafd'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:13.916112 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 989/1967] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:13 2022] DELETE /identity/v3/users/3ead0122048e4b6f99bc5542c5ec0502 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:13.919366 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-45b1a35b-bdb7-4014-80ca-d00af8fbbafd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c55dac7229641849299cf317ab34113', 'request_id': 'req-45b1a35b-bdb7-4014-80ca-d00af8fbbafd'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:13.936556 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-45b1a35b-bdb7-4014-80ca-d00af8fbbafd admin admin] The token cache is being invalidate because project 0c55dac7229641849299cf317ab34113 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:13.936988 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-45b1a35b-bdb7-4014-80ca-d00af8fbbafd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:13.937964 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-45b1a35b-bdb7-4014-80ca-d00af8fbbafd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:13.940394 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 979/1968] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:13 2022] DELETE /identity/v3/projects/0c55dac7229641849299cf317ab34113 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:05:13.945047 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df3be59-c5af-417b-a5c1-c721f86afd39 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:13.947454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4df3be59-c5af-417b-a5c1-c721f86afd39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:14.040080 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b1a35b-bdb7-4014-80ca-d00af8fbbafd admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:14.042077 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-45b1a35b-bdb7-4014-80ca-d00af8fbbafd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:14.123317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df3be59-c5af-417b-a5c1-c721f86afd39 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:14.133747 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df5e07-5596-4a24-b7b0-e22c4e4b7819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:14.140052 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df5e07-5596-4a24-b7b0-e22c4e4b7819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:14.140674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df5e07-5596-4a24-b7b0-e22c4e4b7819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:14.141117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df5e07-5596-4a24-b7b0-e22c4e4b7819 admin admin] PATH_INFO: `/v3/users/dd2d054d4a76428385df2a856f3dfe9e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:14.145807 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df5e07-5596-4a24-b7b0-e22c4e4b7819 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd2d054d4a76428385df2a856f3dfe9e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:14.154942 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df5e07-5596-4a24-b7b0-e22c4e4b7819 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:14.263833 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b1a35b-bdb7-4014-80ca-d00af8fbbafd admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:14.270965 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dafe1d-3dfc-46a8-a41b-08f668701b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:14.271970 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dafe1d-3dfc-46a8-a41b-08f668701b15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:14.272121 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-10df5e07-5596-4a24-b7b0-e22c4e4b7819 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd2d054d4a76428385df2a856f3dfe9e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:14.272557 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dafe1d-3dfc-46a8-a41b-08f668701b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:14.272825 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-10df5e07-5596-4a24-b7b0-e22c4e4b7819 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd2d054d4a76428385df2a856f3dfe9e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:14.272932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dafe1d-3dfc-46a8-a41b-08f668701b15 admin admin] PATH_INFO: `/v3/projects/8ee4156be29e4784bc71a50642807dbb` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:14.273440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-10df5e07-5596-4a24-b7b0-e22c4e4b7819 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd2d054d4a76428385df2a856f3dfe9e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:14.280656 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dafe1d-3dfc-46a8-a41b-08f668701b15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ee4156be29e4784bc71a50642807dbb)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:14.287866 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-10df5e07-5596-4a24-b7b0-e22c4e4b7819 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd2d054d4a76428385df2a856f3dfe9e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:14.295272 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dafe1d-3dfc-46a8-a41b-08f668701b15 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:14.301707 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-10df5e07-5596-4a24-b7b0-e22c4e4b7819 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd2d054d4a76428385df2a856f3dfe9e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:14.313595 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:14.333065 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 990/1969] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:13 2022] DELETE /identity/v3/users/dd2d054d4a76428385df2a856f3dfe9e => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:14.370297 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:14.408811 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-81dafe1d-3dfc-46a8-a41b-08f668701b15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ee4156be29e4784bc71a50642807dbb', 'request_id': 'req-81dafe1d-3dfc-46a8-a41b-08f668701b15'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:14.430852 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-81dafe1d-3dfc-46a8-a41b-08f668701b15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ee4156be29e4784bc71a50642807dbb', 'request_id': 'req-81dafe1d-3dfc-46a8-a41b-08f668701b15'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:14.458042 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-81dafe1d-3dfc-46a8-a41b-08f668701b15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ee4156be29e4784bc71a50642807dbb', 'request_id': 'req-81dafe1d-3dfc-46a8-a41b-08f668701b15'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:14.490598 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-81dafe1d-3dfc-46a8-a41b-08f668701b15 admin admin] The token cache is being invalidate because project 8ee4156be29e4784bc71a50642807dbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:14.490839 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-81dafe1d-3dfc-46a8-a41b-08f668701b15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:14.492314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-81dafe1d-3dfc-46a8-a41b-08f668701b15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:14.503046 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403f923-3ba6-4d0d-854b-974bf1c117f6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:14.503792 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403f923-3ba6-4d0d-854b-974bf1c117f6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:14.504068 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403f923-3ba6-4d0d-854b-974bf1c117f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:14.505753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 980/1970] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:14 2022] DELETE /identity/v3/projects/8ee4156be29e4784bc71a50642807dbb => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:14.549862 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a403f923-3ba6-4d0d-854b-974bf1c117f6 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:14.553710 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a403f923-3ba6-4d0d-854b-974bf1c117f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:14.640423 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dafe1d-3dfc-46a8-a41b-08f668701b15 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:14.642442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-81dafe1d-3dfc-46a8-a41b-08f668701b15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:14.662228 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 991/1971] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:14.673569 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0921d2c8-9012-4aa7-90df-90a3106cc0ba None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:14.674128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0921d2c8-9012-4aa7-90df-90a3106cc0ba None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:14.675253 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0921d2c8-9012-4aa7-90df-90a3106cc0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:14.705515 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0921d2c8-9012-4aa7-90df-90a3106cc0ba None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:14.708585 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0921d2c8-9012-4aa7-90df-90a3106cc0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:14.718203 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 992/1972] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:14.722182 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0921d2c8-9012-4aa7-90df-90a3106cc0ba None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:14.742442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0921d2c8-9012-4aa7-90df-90a3106cc0ba None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:14.743386 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dafe1d-3dfc-46a8-a41b-08f668701b15 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:14.748999 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878fc618-2d52-4937-9044-7744bd5b8bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:14.750036 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878fc618-2d52-4937-9044-7744bd5b8bb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:14.750309 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878fc618-2d52-4937-9044-7744bd5b8bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:14.750599 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878fc618-2d52-4937-9044-7744bd5b8bb6 admin admin] PATH_INFO: `/v3/projects/1fa2bceb192e4817a5ba676203c47dbd` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:14.752290 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878fc618-2d52-4937-9044-7744bd5b8bb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fa2bceb192e4817a5ba676203c47dbd)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:14.755729 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878fc618-2d52-4937-9044-7744bd5b8bb6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:14.765014 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad695be2-d24f-49d7-8b8f-e702204c504d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:14.765862 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad695be2-d24f-49d7-8b8f-e702204c504d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:14.766037 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad695be2-d24f-49d7-8b8f-e702204c504d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:14.766271 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad695be2-d24f-49d7-8b8f-e702204c504d admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:14.767010 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad695be2-d24f-49d7-8b8f-e702204c504d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:05:14.767321 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad695be2-d24f-49d7-8b8f-e702204c504d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:14.768940 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad695be2-d24f-49d7-8b8f-e702204c504d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:14.787746 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 993/1973] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 16:05:14.793031 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad695be2-d24f-49d7-8b8f-e702204c504d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:14.806593 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad695be2-d24f-49d7-8b8f-e702204c504d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:14.829193 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-878fc618-2d52-4937-9044-7744bd5b8bb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fa2bceb192e4817a5ba676203c47dbd', 'request_id': 'req-878fc618-2d52-4937-9044-7744bd5b8bb6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:14.843582 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-878fc618-2d52-4937-9044-7744bd5b8bb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fa2bceb192e4817a5ba676203c47dbd', 'request_id': 'req-878fc618-2d52-4937-9044-7744bd5b8bb6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:14.853200 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-878fc618-2d52-4937-9044-7744bd5b8bb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fa2bceb192e4817a5ba676203c47dbd', 'request_id': 'req-878fc618-2d52-4937-9044-7744bd5b8bb6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:14.867611 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-878fc618-2d52-4937-9044-7744bd5b8bb6 admin admin] The token cache is being invalidate because project 1fa2bceb192e4817a5ba676203c47dbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:14.867996 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-878fc618-2d52-4937-9044-7744bd5b8bb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:14.868751 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-878fc618-2d52-4937-9044-7744bd5b8bb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:14.870915 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 981/1974] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:14 2022] DELETE /identity/v3/projects/1fa2bceb192e4817a5ba676203c47dbd => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:14.875591 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6fae91-aa2c-4801-aca0-17e23ba606f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:14.876246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac37bd7-45b2-4c9f-a927-d41123afcfb3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:14.876617 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac37bd7-45b2-4c9f-a927-d41123afcfb3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:14.876777 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fae91-aa2c-4801-aca0-17e23ba606f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:14.877200 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fae91-aa2c-4801-aca0-17e23ba606f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:14.877652 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fae91-aa2c-4801-aca0-17e23ba606f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:14.878065 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac37bd7-45b2-4c9f-a927-d41123afcfb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:14.878705 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6fae91-aa2c-4801-aca0-17e23ba606f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:14.881366 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6fae91-aa2c-4801-aca0-17e23ba606f5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:14.901697 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-eac37bd7-45b2-4c9f-a927-d41123afcfb3 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:14.903266 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-eac37bd7-45b2-4c9f-a927-d41123afcfb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:14.922233 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 982/1975] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:14.924777 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015d163-8597-4d75-8189-bceca3662d67 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:14.925315 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015d163-8597-4d75-8189-bceca3662d67 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:14.925442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 994/1976] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:14 2022] POST /identity/v3/projects => generated 373 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:05:14.925548 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015d163-8597-4d75-8189-bceca3662d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:14.930817 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6fae91-aa2c-4801-aca0-17e23ba606f5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:14.932655 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8a6fae91-aa2c-4801-aca0-17e23ba606f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:14.949351 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4015d163-8597-4d75-8189-bceca3662d67 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:14.950547 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4015d163-8597-4d75-8189-bceca3662d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:14.994640 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:15.002712 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:15.008075 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6fae91-aa2c-4801-aca0-17e23ba606f5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:15.017202 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fed454-61b9-4898-87f1-8bd65c312bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:15.018217 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fed454-61b9-4898-87f1-8bd65c312bc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:15.019099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fed454-61b9-4898-87f1-8bd65c312bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:15.019624 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fed454-61b9-4898-87f1-8bd65c312bc3 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:15.022174 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fed454-61b9-4898-87f1-8bd65c312bc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:15.025152 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fed454-61b9-4898-87f1-8bd65c312bc3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:15.037127 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 983/1977] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:15.038923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015d163-8597-4d75-8189-bceca3662d67 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:15.048973 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015d163-8597-4d75-8189-bceca3662d67 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:15.053999 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d9311-8bcd-4f54-9d60-803499505ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:15.054760 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d9311-8bcd-4f54-9d60-803499505ca8 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:15.055118 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d9311-8bcd-4f54-9d60-803499505ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:15.055475 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d9311-8bcd-4f54-9d60-803499505ca8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:15.056292 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d9311-8bcd-4f54-9d60-803499505ca8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:05:15.056676 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 995/1978] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:14 2022] POST /identity/v3/users => generated 287 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:15.056799 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d9311-8bcd-4f54-9d60-803499505ca8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:15.058574 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d9311-8bcd-4f54-9d60-803499505ca8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:15.061009 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fed454-61b9-4898-87f1-8bd65c312bc3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:15.072694 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fed454-61b9-4898-87f1-8bd65c312bc3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:15.073156 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 984/1979] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:05:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:05:15.077313 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d9311-8bcd-4f54-9d60-803499505ca8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:15.079245 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0f4d9311-8bcd-4f54-9d60-803499505ca8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:15.080639 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b59cbbc-c98e-4ae4-bd4a-3d68ad1e50c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:15.081323 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59cbbc-c98e-4ae4-bd4a-3d68ad1e50c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:15.081635 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59cbbc-c98e-4ae4-bd4a-3d68ad1e50c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:15.082166 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59cbbc-c98e-4ae4-bd4a-3d68ad1e50c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:15.082704 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b59cbbc-c98e-4ae4-bd4a-3d68ad1e50c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:15.084494 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b59cbbc-c98e-4ae4-bd4a-3d68ad1e50c3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:15.096119 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 996/1980] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 16:05:15.096634 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d9311-8bcd-4f54-9d60-803499505ca8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:15.099089 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b59cbbc-c98e-4ae4-bd4a-3d68ad1e50c3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:15.103476 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861c796-864e-4850-8da4-f0e253e4d57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:15.104510 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861c796-864e-4850-8da4-f0e253e4d57a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:15.104811 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861c796-864e-4850-8da4-f0e253e4d57a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:15.105090 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861c796-864e-4850-8da4-f0e253e4d57a admin admin] PATH_INFO: `/v3/projects/48cc9814518d414aa75786a356915354` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:15.106896 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861c796-864e-4850-8da4-f0e253e4d57a admin admin] RBAC: Authorizing `identity:delete_project(project_id=48cc9814518d414aa75786a356915354)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:15.109925 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861c796-864e-4850-8da4-f0e253e4d57a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:15.120285 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b59cbbc-c98e-4ae4-bd4a-3d68ad1e50c3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:15.125826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8edf069-85c1-43c2-b621-dcc53aa00d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:15.126497 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8edf069-85c1-43c2-b621-dcc53aa00d79 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:15.126786 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8edf069-85c1-43c2-b621-dcc53aa00d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:15.127073 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8edf069-85c1-43c2-b621-dcc53aa00d79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:15.127778 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8edf069-85c1-43c2-b621-dcc53aa00d79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:15.129849 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8edf069-85c1-43c2-b621-dcc53aa00d79 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:15.169753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 997/1981] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:15 2022] POST /identity/v3/projects => generated 385 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:15.170735 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8edf069-85c1-43c2-b621-dcc53aa00d79 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:15.177969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f861c796-864e-4850-8da4-f0e253e4d57a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48cc9814518d414aa75786a356915354', 'request_id': 'req-f861c796-864e-4850-8da4-f0e253e4d57a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:15.185647 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f861c796-864e-4850-8da4-f0e253e4d57a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48cc9814518d414aa75786a356915354', 'request_id': 'req-f861c796-864e-4850-8da4-f0e253e4d57a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:15.195256 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f861c796-864e-4850-8da4-f0e253e4d57a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48cc9814518d414aa75786a356915354', 'request_id': 'req-f861c796-864e-4850-8da4-f0e253e4d57a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:15.207778 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f861c796-864e-4850-8da4-f0e253e4d57a admin admin] The token cache is being invalidate because project 48cc9814518d414aa75786a356915354 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:15.208070 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f861c796-864e-4850-8da4-f0e253e4d57a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:15.208718 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f861c796-864e-4850-8da4-f0e253e4d57a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:15.212223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 985/1982] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:15 2022] DELETE /identity/v3/projects/48cc9814518d414aa75786a356915354 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:15.212942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861c796-864e-4850-8da4-f0e253e4d57a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:15.214350 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f861c796-864e-4850-8da4-f0e253e4d57a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:15.245989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8edf069-85c1-43c2-b621-dcc53aa00d79 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:15.252488 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e09b54-6125-4a1e-8835-01743359096d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:15.255122 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e09b54-6125-4a1e-8835-01743359096d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:15.255437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e09b54-6125-4a1e-8835-01743359096d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:15.256024 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e09b54-6125-4a1e-8835-01743359096d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:15.256600 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e09b54-6125-4a1e-8835-01743359096d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:15.258785 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e09b54-6125-4a1e-8835-01743359096d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:15.272009 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 998/1983] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:15.274227 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29033c42-24f0-4a0a-883c-0f6313d8a0a9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:15.274824 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29033c42-24f0-4a0a-883c-0f6313d8a0a9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:15.274939 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29033c42-24f0-4a0a-883c-0f6313d8a0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:15.320373 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-29033c42-24f0-4a0a-883c-0f6313d8a0a9 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:15.321790 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-29033c42-24f0-4a0a-883c-0f6313d8a0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:15.331692 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861c796-864e-4850-8da4-f0e253e4d57a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:15.337242 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ade8bd-2db3-4a8f-a5d3-4f9eb109f98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:15.337786 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 999/1984] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:15.338989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29033c42-24f0-4a0a-883c-0f6313d8a0a9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:15.340378 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-29033c42-24f0-4a0a-883c-0f6313d8a0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:15.342287 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ade8bd-2db3-4a8f-a5d3-4f9eb109f98f admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:15.342740 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ade8bd-2db3-4a8f-a5d3-4f9eb109f98f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:15.343155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ade8bd-2db3-4a8f-a5d3-4f9eb109f98f admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:15.344257 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ade8bd-2db3-4a8f-a5d3-4f9eb109f98f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:15.346494 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ade8bd-2db3-4a8f-a5d3-4f9eb109f98f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:15.354260 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29033c42-24f0-4a0a-883c-0f6313d8a0a9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:15.360860 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301c02f-a993-48e7-b3fc-86716bea4b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:15.361913 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301c02f-a993-48e7-b3fc-86716bea4b4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:15.362210 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301c02f-a993-48e7-b3fc-86716bea4b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:15.362491 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301c02f-a993-48e7-b3fc-86716bea4b4c admin admin] PATH_INFO: `/v3/projects/f49a5f13c3ef42b8aa5a4f9a38ca9b76/users/8c28991387694c958cdd7a98e4ce12ae/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:15.369480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 986/1985] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:15 2022] POST /identity/v3/users => generated 293 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:15.371094 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c8899-a876-4f49-9785-421e05710ff4 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:15.371462 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c8899-a876-4f49-9785-421e05710ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:15.371820 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c8899-a876-4f49-9785-421e05710ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:15.383765 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1301c02f-a993-48e7-b3fc-86716bea4b4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f49a5f13c3ef42b8aa5a4f9a38ca9b76, user_id=8c28991387694c958cdd7a98e4ce12ae, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:15.386884 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1301c02f-a993-48e7-b3fc-86716bea4b4c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:15.393120 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5e7c8899-a876-4f49-9785-421e05710ff4 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:15.394698 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5e7c8899-a876-4f49-9785-421e05710ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:15.404185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 987/1986] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:15.405176 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c8899-a876-4f49-9785-421e05710ff4 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:15.411832 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1000/1987] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:15 2022] PUT /identity/v3/projects/f49a5f13c3ef42b8aa5a4f9a38ca9b76/users/8c28991387694c958cdd7a98e4ce12ae/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:15.412559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301c02f-a993-48e7-b3fc-86716bea4b4c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:15.419932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c8899-a876-4f49-9785-421e05710ff4 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:15.461423 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:15.498530 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301c02f-a993-48e7-b3fc-86716bea4b4c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:15.498763 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0dab3-b260-4bff-aaef-2b15872ebeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:15.499759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0dab3-b260-4bff-aaef-2b15872ebeb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:15.500044 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0dab3-b260-4bff-aaef-2b15872ebeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:15.500327 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0dab3-b260-4bff-aaef-2b15872ebeb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:15.501209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d0dab3-b260-4bff-aaef-2b15872ebeb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:15.503532 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d0dab3-b260-4bff-aaef-2b15872ebeb6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:15.504413 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658c5872-1f4a-405e-83f5-7c4b3109fbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:15.505193 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c5872-1f4a-405e-83f5-7c4b3109fbe5 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:15.505674 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c5872-1f4a-405e-83f5-7c4b3109fbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:15.505998 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c5872-1f4a-405e-83f5-7c4b3109fbe5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:15.506851 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658c5872-1f4a-405e-83f5-7c4b3109fbe5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:05:15.507952 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658c5872-1f4a-405e-83f5-7c4b3109fbe5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:15.509831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658c5872-1f4a-405e-83f5-7c4b3109fbe5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:15.519778 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 988/1988] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:15.521279 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b9b55-14c4-4cfc-a081-a9fb2480d513 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:15.521742 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b9b55-14c4-4cfc-a081-a9fb2480d513 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:15.522242 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b9b55-14c4-4cfc-a081-a9fb2480d513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:15.526998 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1001/1989] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:05:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:05:15.527609 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658c5872-1f4a-405e-83f5-7c4b3109fbe5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:15.544703 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7f4b9b55-14c4-4cfc-a081-a9fb2480d513 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:15.545820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658c5872-1f4a-405e-83f5-7c4b3109fbe5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:15.546231 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7f4b9b55-14c4-4cfc-a081-a9fb2480d513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:15.556777 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b731ed1-5a75-4cf2-b9fd-697150a26423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:15.557460 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 989/1990] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:15.558338 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b9b55-14c4-4cfc-a081-a9fb2480d513 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:15.560710 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b731ed1-5a75-4cf2-b9fd-697150a26423 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:15.560988 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b731ed1-5a75-4cf2-b9fd-697150a26423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:15.561277 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b731ed1-5a75-4cf2-b9fd-697150a26423 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:15.562248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b731ed1-5a75-4cf2-b9fd-697150a26423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:15.564034 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b731ed1-5a75-4cf2-b9fd-697150a26423 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:15.571831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:15.578341 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b9b55-14c4-4cfc-a081-a9fb2480d513 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:15.584788 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1002/1991] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:15.584788 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b731ed1-5a75-4cf2-b9fd-697150a26423 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:15.585785 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c3b5c4-9e08-40d7-bb3e-5a5c6b9d3269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:15.586241 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2b731ed1-5a75-4cf2-b9fd-697150a26423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:15.587615 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c3b5c4-9e08-40d7-bb3e-5a5c6b9d3269 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:15.590022 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c3b5c4-9e08-40d7-bb3e-5a5c6b9d3269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:15.590476 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c3b5c4-9e08-40d7-bb3e-5a5c6b9d3269 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:15.591641 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c3b5c4-9e08-40d7-bb3e-5a5c6b9d3269 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:15.596050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c3b5c4-9e08-40d7-bb3e-5a5c6b9d3269 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:15.639123 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 990/1992] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:15 2022] POST /identity/v3/projects => generated 383 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:15.639971 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c3b5c4-9e08-40d7-bb3e-5a5c6b9d3269 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:15.688418 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b731ed1-5a75-4cf2-b9fd-697150a26423 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:15.720766 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c3b5c4-9e08-40d7-bb3e-5a5c6b9d3269 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:15.726776 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0726e2fe-ea4a-4c96-9cf9-c6b8eeb45ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:15.727634 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726e2fe-ea4a-4c96-9cf9-c6b8eeb45ac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:15.727717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726e2fe-ea4a-4c96-9cf9-c6b8eeb45ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:15.727999 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726e2fe-ea4a-4c96-9cf9-c6b8eeb45ac5 admin admin] PATH_INFO: `/v3/projects/328d804f3dfe4aeca17981877380bcde/users/aade5ac32498406db53beaf6d87ba3a6/roles/54195e660fc349978b39d5794765a095` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:15.745435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0726e2fe-ea4a-4c96-9cf9-c6b8eeb45ac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=328d804f3dfe4aeca17981877380bcde, user_id=aade5ac32498406db53beaf6d87ba3a6, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:15.748416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0726e2fe-ea4a-4c96-9cf9-c6b8eeb45ac5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:15.766843 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 991/1993] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:15 2022] PUT /identity/v3/projects/328d804f3dfe4aeca17981877380bcde/users/aade5ac32498406db53beaf6d87ba3a6/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:15.767904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0726e2fe-ea4a-4c96-9cf9-c6b8eeb45ac5 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:15.778309 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a936d506-dc18-4cc7-8728-8bd492183b58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:15.778557 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936d506-dc18-4cc7-8728-8bd492183b58 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:15.779233 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936d506-dc18-4cc7-8728-8bd492183b58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:15.779585 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936d506-dc18-4cc7-8728-8bd492183b58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:15.794546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a936d506-dc18-4cc7-8728-8bd492183b58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:15.796290 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a936d506-dc18-4cc7-8728-8bd492183b58 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:15.811842 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1003/1994] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:15.813219 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1702ca0c-42d4-4427-8195-e2be89692c04 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:15.813461 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1702ca0c-42d4-4427-8195-e2be89692c04 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:15.813791 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1702ca0c-42d4-4427-8195-e2be89692c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:15.850393 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1702ca0c-42d4-4427-8195-e2be89692c04 None None] MFA Rules not processed for user `aade5ac32498406db53beaf6d87ba3a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:15.852849 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1702ca0c-42d4-4427-8195-e2be89692c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:15.868558 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0726e2fe-ea4a-4c96-9cf9-c6b8eeb45ac5 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:15.873968 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e8247-30ae-462d-a864-7ac9a66ebf7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:15.874738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e8247-30ae-462d-a864-7ac9a66ebf7c admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:15.875155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e8247-30ae-462d-a864-7ac9a66ebf7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:15.875915 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e8247-30ae-462d-a864-7ac9a66ebf7c admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:15.877080 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e8247-30ae-462d-a864-7ac9a66ebf7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:15.879368 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e8247-30ae-462d-a864-7ac9a66ebf7c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:15.907226 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 992/1995] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:15 2022] POST /identity/v3/users => generated 292 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:15.912437 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e8247-30ae-462d-a864-7ac9a66ebf7c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:15.929926 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e8247-30ae-462d-a864-7ac9a66ebf7c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:15.934949 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20374b09-d494-4722-aa7c-136d4e04e86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:15.935555 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20374b09-d494-4722-aa7c-136d4e04e86a admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:15.935976 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20374b09-d494-4722-aa7c-136d4e04e86a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:15.936262 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20374b09-d494-4722-aa7c-136d4e04e86a admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:15.937009 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20374b09-d494-4722-aa7c-136d4e04e86a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:15.938665 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20374b09-d494-4722-aa7c-136d4e04e86a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:15.958530 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1004/1996] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:15.969351 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1702ca0c-42d4-4427-8195-e2be89692c04 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:15.969670 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 993/1997] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 16:05:15.970262 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20374b09-d494-4722-aa7c-136d4e04e86a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:15.977621 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1702ca0c-42d4-4427-8195-e2be89692c04 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:15.978471 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:15.989377 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e256cae-e1ec-4063-aebd-e60c1b46001c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:15.991699 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e256cae-e1ec-4063-aebd-e60c1b46001c admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:15.992187 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e256cae-e1ec-4063-aebd-e60c1b46001c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:15.992620 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e256cae-e1ec-4063-aebd-e60c1b46001c admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:15.993792 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e256cae-e1ec-4063-aebd-e60c1b46001c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:15.996307 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e256cae-e1ec-4063-aebd-e60c1b46001c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:16.003473 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20374b09-d494-4722-aa7c-136d4e04e86a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:16.015649 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d806359-d4ed-4931-82eb-520d43d93465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:16.016233 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d806359-d4ed-4931-82eb-520d43d93465 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:16.016604 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d806359-d4ed-4931-82eb-520d43d93465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:16.017031 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d806359-d4ed-4931-82eb-520d43d93465 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:16.018018 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d806359-d4ed-4931-82eb-520d43d93465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:16.019934 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d806359-d4ed-4931-82eb-520d43d93465 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:16.034170 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 994/1998] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:16.040011 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d806359-d4ed-4931-82eb-520d43d93465 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:16.047122 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1005/1999] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:15 2022] POST /identity/v3/projects => generated 385 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:16.051787 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e256cae-e1ec-4063-aebd-e60c1b46001c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:16.057943 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d806359-d4ed-4931-82eb-520d43d93465 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:16.137221 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:16.181614 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf9f51-48a1-4f23-b827-11c3c3c10784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:16.182659 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf9f51-48a1-4f23-b827-11c3c3c10784 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:16.183334 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf9f51-48a1-4f23-b827-11c3c3c10784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:16.183677 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf9f51-48a1-4f23-b827-11c3c3c10784 admin admin] PATH_INFO: `/v3/projects/b5bc6434f5f5444db3ad35b3f33c5089/users/9f29058d936b41be9273be9559b0269e/roles/54195e660fc349978b39d5794765a095` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:16.202893 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e256cae-e1ec-4063-aebd-e60c1b46001c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:16.215762 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf9f51-48a1-4f23-b827-11c3c3c10784 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5bc6434f5f5444db3ad35b3f33c5089, user_id=9f29058d936b41be9273be9559b0269e, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:16.217088 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cdd534-1c6f-423d-a926-231fa3c75e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:16.217985 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cdd534-1c6f-423d-a926-231fa3c75e94 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:16.218342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cdd534-1c6f-423d-a926-231fa3c75e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:16.218626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cdd534-1c6f-423d-a926-231fa3c75e94 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:16.219480 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cdd534-1c6f-423d-a926-231fa3c75e94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:16.221696 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cdd534-1c6f-423d-a926-231fa3c75e94 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:16.223807 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf9f51-48a1-4f23-b827-11c3c3c10784 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:16.249727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 995/2000] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:16 2022] PUT /identity/v3/projects/b5bc6434f5f5444db3ad35b3f33c5089/users/9f29058d936b41be9273be9559b0269e/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:16.260779 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b242a93-0457-4b5e-ada4-e449dbf2d594 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:16.261043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b242a93-0457-4b5e-ada4-e449dbf2d594 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:16.262946 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b242a93-0457-4b5e-ada4-e449dbf2d594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:16.271942 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1006/2001] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:16 2022] POST /identity/v3/users => generated 299 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:16.275625 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cdd534-1c6f-423d-a926-231fa3c75e94 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:16.288370 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cdd534-1c6f-423d-a926-231fa3c75e94 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:16.312039 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1b242a93-0457-4b5e-ada4-e449dbf2d594 None None] MFA Rules not processed for user `9f29058d936b41be9273be9559b0269e`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:16.314025 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1b242a93-0457-4b5e-ada4-e449dbf2d594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:16.373026 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e5e732-a996-4b97-9856-51ab8d0a39eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:16.373979 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5e732-a996-4b97-9856-51ab8d0a39eb admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:16.374565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5e732-a996-4b97-9856-51ab8d0a39eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:16.375145 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5e732-a996-4b97-9856-51ab8d0a39eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:16.376419 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e5e732-a996-4b97-9856-51ab8d0a39eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:16.378811 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e5e732-a996-4b97-9856-51ab8d0a39eb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:16.393443 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1007/2002] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:16.398295 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e5e732-a996-4b97-9856-51ab8d0a39eb admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:16.411303 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e5e732-a996-4b97-9856-51ab8d0a39eb admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:16.414072 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 996/2003] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:16.419237 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b242a93-0457-4b5e-ada4-e449dbf2d594 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:16.420267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e80c20-cb06-47d7-893c-66661eef71e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:16.421349 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e80c20-cb06-47d7-893c-66661eef71e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:16.421657 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e80c20-cb06-47d7-893c-66661eef71e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:16.421982 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e80c20-cb06-47d7-893c-66661eef71e8 admin admin] PATH_INFO: `/v3/projects/d1438a58f975415987bd91e7e93608ae/users/2017826094f443ba9e6a62610b6773f8/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:16.434995 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b242a93-0457-4b5e-ada4-e449dbf2d594 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:16.440133 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e80c20-cb06-47d7-893c-66661eef71e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1438a58f975415987bd91e7e93608ae, user_id=2017826094f443ba9e6a62610b6773f8, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:16.440971 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eaaeeb-9a53-4524-9629-2080130b7a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:16.441756 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eaaeeb-9a53-4524-9629-2080130b7a2f admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:16.442092 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eaaeeb-9a53-4524-9629-2080130b7a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:16.443015 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eaaeeb-9a53-4524-9629-2080130b7a2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:16.443175 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e80c20-cb06-47d7-893c-66661eef71e8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:16.444041 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eaaeeb-9a53-4524-9629-2080130b7a2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:16.446848 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eaaeeb-9a53-4524-9629-2080130b7a2f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:16.465045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1008/2004] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:16 2022] PUT /identity/v3/projects/d1438a58f975415987bd91e7e93608ae/users/2017826094f443ba9e6a62610b6773f8/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:16.468883 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e80c20-cb06-47d7-893c-66661eef71e8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:16.488116 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 997/2005] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:16 2022] POST /identity/v3/projects => generated 379 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:16.492684 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eaaeeb-9a53-4524-9629-2080130b7a2f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:16.556311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e80c20-cb06-47d7-893c-66661eef71e8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:16.574043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:16.600527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:16.605287 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eaaeeb-9a53-4524-9629-2080130b7a2f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:16.610586 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32865dd0-1c6f-4a03-bf5e-d7f96345944a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:16.614202 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32865dd0-1c6f-4a03-bf5e-d7f96345944a admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:16.614898 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32865dd0-1c6f-4a03-bf5e-d7f96345944a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:16.616341 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32865dd0-1c6f-4a03-bf5e-d7f96345944a admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:16.619430 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32865dd0-1c6f-4a03-bf5e-d7f96345944a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:16.624109 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32865dd0-1c6f-4a03-bf5e-d7f96345944a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:16.646399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3b786-82e8-47ea-9c6b-39c0a68d6205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:16.647095 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3b786-82e8-47ea-9c6b-39c0a68d6205 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:16.647610 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3b786-82e8-47ea-9c6b-39c0a68d6205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:16.647894 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3b786-82e8-47ea-9c6b-39c0a68d6205 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:16.648986 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d3b786-82e8-47ea-9c6b-39c0a68d6205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:16.650551 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d3b786-82e8-47ea-9c6b-39c0a68d6205 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:16.657822 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 998/2006] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:16 2022] POST /identity/v3/users => generated 296 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:05:16.661724 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32865dd0-1c6f-4a03-bf5e-d7f96345944a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:16.665396 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1009/2007] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:16.666392 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3b786-82e8-47ea-9c6b-39c0a68d6205 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:16.679836 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32865dd0-1c6f-4a03-bf5e-d7f96345944a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:16.680240 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3b786-82e8-47ea-9c6b-39c0a68d6205 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:16.686798 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da04c789-40e3-4dbb-a245-542a86f1812b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:16.687652 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdff407-85eb-476e-ae77-144fdb554d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:16.687803 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da04c789-40e3-4dbb-a245-542a86f1812b admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:16.688022 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da04c789-40e3-4dbb-a245-542a86f1812b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:16.688755 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da04c789-40e3-4dbb-a245-542a86f1812b admin admin] PATH_INFO: `/v3/projects/d1438a58f975415987bd91e7e93608ae/users/2017826094f443ba9e6a62610b6773f8/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:16.689099 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdff407-85eb-476e-ae77-144fdb554d5e admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:16.689657 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdff407-85eb-476e-ae77-144fdb554d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:16.690136 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdff407-85eb-476e-ae77-144fdb554d5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:16.691458 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdff407-85eb-476e-ae77-144fdb554d5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:16.691565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da04c789-40e3-4dbb-a245-542a86f1812b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1438a58f975415987bd91e7e93608ae, user_id=2017826094f443ba9e6a62610b6773f8, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:16.693469 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdff407-85eb-476e-ae77-144fdb554d5e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:16.695177 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da04c789-40e3-4dbb-a245-542a86f1812b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:16.707773 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 999/2008] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:16.713863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdff407-85eb-476e-ae77-144fdb554d5e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:16.726844 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1010/2009] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:16 2022] PUT /identity/v3/projects/d1438a58f975415987bd91e7e93608ae/users/2017826094f443ba9e6a62610b6773f8/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:05:16.737650 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671483e2-2b36-457d-8efb-57a099f310d6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:16.738130 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671483e2-2b36-457d-8efb-57a099f310d6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:16.738462 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671483e2-2b36-457d-8efb-57a099f310d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:16.744558 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdff407-85eb-476e-ae77-144fdb554d5e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:16.801983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-671483e2-2b36-457d-8efb-57a099f310d6 None None] MFA Rules not processed for user `2017826094f443ba9e6a62610b6773f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:16.803506 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-671483e2-2b36-457d-8efb-57a099f310d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:16.927959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b32242-8e2f-4f89-ae87-5ba8d734ca4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:16.930118 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b32242-8e2f-4f89-ae87-5ba8d734ca4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:16.930488 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b32242-8e2f-4f89-ae87-5ba8d734ca4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:16.930843 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b32242-8e2f-4f89-ae87-5ba8d734ca4c admin admin] PATH_INFO: `/v3/projects/89ff7c99276c4c24833e67d13daf51d6/users/32386a5e9a804abb880341e256331f9a/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:16.955506 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b32242-8e2f-4f89-ae87-5ba8d734ca4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=89ff7c99276c4c24833e67d13daf51d6, user_id=32386a5e9a804abb880341e256331f9a, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:16.960411 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b32242-8e2f-4f89-ae87-5ba8d734ca4c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:16.977087 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1011/2010] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:16 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:05:16.978739 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4ae89-6706-488f-a70d-9496fade7b08 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:16.979030 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4ae89-6706-488f-a70d-9496fade7b08 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:16.979315 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4ae89-6706-488f-a70d-9496fade7b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:16.993273 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1000/2011] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:16 2022] PUT /identity/v3/projects/89ff7c99276c4c24833e67d13daf51d6/users/32386a5e9a804abb880341e256331f9a/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:16.996067 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b32242-8e2f-4f89-ae87-5ba8d734ca4c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:17.074273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8ca4ae89-6706-488f-a70d-9496fade7b08 None None] MFA Rules not processed for user `8c28991387694c958cdd7a98e4ce12ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:17.075527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8ca4ae89-6706-488f-a70d-9496fade7b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:17.144201 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b32242-8e2f-4f89-ae87-5ba8d734ca4c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:17.158018 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35417a7-31e4-42eb-b7e2-579af15abb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:17.159505 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35417a7-31e4-42eb-b7e2-579af15abb21 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:17.160502 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35417a7-31e4-42eb-b7e2-579af15abb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:17.162111 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35417a7-31e4-42eb-b7e2-579af15abb21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:17.163942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35417a7-31e4-42eb-b7e2-579af15abb21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:17.168658 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35417a7-31e4-42eb-b7e2-579af15abb21 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:17.199488 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1001/2012] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 16:05:17.202425 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35417a7-31e4-42eb-b7e2-579af15abb21 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:17.205703 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e35417a7-31e4-42eb-b7e2-579af15abb21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:17.226970 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1012/2013] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:17.227753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4ae89-6706-488f-a70d-9496fade7b08 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:17.239407 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4ae89-6706-488f-a70d-9496fade7b08 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:17.245116 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c82a1-b957-49cc-8399-010cd7bc56dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:17.246323 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c82a1-b957-49cc-8399-010cd7bc56dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:17.246609 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c82a1-b957-49cc-8399-010cd7bc56dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:17.246892 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c82a1-b957-49cc-8399-010cd7bc56dc admin admin] PATH_INFO: `/v3/projects/89ff7c99276c4c24833e67d13daf51d6/users/32386a5e9a804abb880341e256331f9a/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:17.248982 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c82a1-b957-49cc-8399-010cd7bc56dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=89ff7c99276c4c24833e67d13daf51d6, user_id=32386a5e9a804abb880341e256331f9a, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:17.251978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c82a1-b957-49cc-8399-010cd7bc56dc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:17.268272 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1013/2014] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:17 2022] PUT /identity/v3/projects/89ff7c99276c4c24833e67d13daf51d6/users/32386a5e9a804abb880341e256331f9a/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:17.268272 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c82a1-b957-49cc-8399-010cd7bc56dc admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:17.299298 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:17.305530 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:17.333050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35417a7-31e4-42eb-b7e2-579af15abb21 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:17.368072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c82a1-b957-49cc-8399-010cd7bc56dc admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:17.375421 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0335620-b081-4c1d-90dc-0d11068e91e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:17.376270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0335620-b081-4c1d-90dc-0d11068e91e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:17.376727 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0335620-b081-4c1d-90dc-0d11068e91e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:17.377542 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0335620-b081-4c1d-90dc-0d11068e91e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:17.379148 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0335620-b081-4c1d-90dc-0d11068e91e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:17.381244 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0335620-b081-4c1d-90dc-0d11068e91e7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:17.406094 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69c83d1-668a-40d6-9c12-f125d3611192 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:17.406905 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c83d1-668a-40d6-9c12-f125d3611192 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:17.407197 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c83d1-668a-40d6-9c12-f125d3611192 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:17.407560 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c83d1-668a-40d6-9c12-f125d3611192 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:17.421622 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1014/2015] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:17 2022] POST /identity/v3/projects => generated 373 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:17.426331 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd8cdf-3911-4acc-acfb-b861636b1e6d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:17.426729 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd8cdf-3911-4acc-acfb-b861636b1e6d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:17.427086 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd8cdf-3911-4acc-acfb-b861636b1e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:17.461024 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-13fd8cdf-3911-4acc-acfb-b861636b1e6d None None] MFA Rules not processed for user `32386a5e9a804abb880341e256331f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:17.462534 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-13fd8cdf-3911-4acc-acfb-b861636b1e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:17.511339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69c83d1-668a-40d6-9c12-f125d3611192 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:17.513883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69c83d1-668a-40d6-9c12-f125d3611192 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:17.528672 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1002/2016] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:17 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:17.529702 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69c83d1-668a-40d6-9c12-f125d3611192 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:17.542914 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69c83d1-668a-40d6-9c12-f125d3611192 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:17.576625 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1015/2017] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:17 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:17.592817 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd8cdf-3911-4acc-acfb-b861636b1e6d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:17.643230 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39275d00-fe05-4012-b4a8-a21eea58e411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:17.644258 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39275d00-fe05-4012-b4a8-a21eea58e411 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:17.644688 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39275d00-fe05-4012-b4a8-a21eea58e411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:17.644989 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39275d00-fe05-4012-b4a8-a21eea58e411 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:17.645727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39275d00-fe05-4012-b4a8-a21eea58e411 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:17.649417 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39275d00-fe05-4012-b4a8-a21eea58e411 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:17.673478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd8cdf-3911-4acc-acfb-b861636b1e6d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:17.677725 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1003/2018] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:17 2022] POST /identity/v3/users => generated 293 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:17.681890 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39275d00-fe05-4012-b4a8-a21eea58e411 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:17.683540 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614f2ec-508d-417c-8504-214be75d9a83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:17.686719 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614f2ec-508d-417c-8504-214be75d9a83 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:17.687273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614f2ec-508d-417c-8504-214be75d9a83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:17.687710 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614f2ec-508d-417c-8504-214be75d9a83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:17.698323 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39275d00-fe05-4012-b4a8-a21eea58e411 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:17.707858 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff3a41-f4e1-48a6-88bb-ca016eea388b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:17.710950 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff3a41-f4e1-48a6-88bb-ca016eea388b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:17.710950 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff3a41-f4e1-48a6-88bb-ca016eea388b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:17.710950 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff3a41-f4e1-48a6-88bb-ca016eea388b admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:17.710950 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff3a41-f4e1-48a6-88bb-ca016eea388b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:17.712800 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff3a41-f4e1-48a6-88bb-ca016eea388b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:17.714943 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5614f2ec-508d-417c-8504-214be75d9a83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:17.720789 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5614f2ec-508d-417c-8504-214be75d9a83 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:17.724929 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1004/2019] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:17.728978 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff3a41-f4e1-48a6-88bb-ca016eea388b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:17.733259 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1016/2020] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:17 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:17.741048 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff3a41-f4e1-48a6-88bb-ca016eea388b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:17.747444 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98061e2-8db5-4c6e-b1d3-62f763883d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:17.748648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98061e2-8db5-4c6e-b1d3-62f763883d03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:17.749010 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98061e2-8db5-4c6e-b1d3-62f763883d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:17.749366 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98061e2-8db5-4c6e-b1d3-62f763883d03 admin admin] PATH_INFO: `/v3/projects/f66a04ebde8a4e628b700eee4ea2448f/users/95d1419f03104742a2bd3c56a840112e/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:17.772107 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98061e2-8db5-4c6e-b1d3-62f763883d03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f66a04ebde8a4e628b700eee4ea2448f, user_id=95d1419f03104742a2bd3c56a840112e, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:17.776847 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98061e2-8db5-4c6e-b1d3-62f763883d03 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:17.791385 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:17.798364 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1005/2021] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:17 2022] PUT /identity/v3/projects/f66a04ebde8a4e628b700eee4ea2448f/users/95d1419f03104742a2bd3c56a840112e/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:17.802409 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614f2ec-508d-417c-8504-214be75d9a83 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:17.815276 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614f2ec-508d-417c-8504-214be75d9a83 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:17.845513 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98061e2-8db5-4c6e-b1d3-62f763883d03 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:17.889856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:17.929647 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7518ff8-8f30-484e-8cbc-0626893864ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:17.931794 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7518ff8-8f30-484e-8cbc-0626893864ad admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:17.933976 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7518ff8-8f30-484e-8cbc-0626893864ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:17.934301 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7518ff8-8f30-484e-8cbc-0626893864ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:17.935226 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7518ff8-8f30-484e-8cbc-0626893864ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:17.935783 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98061e2-8db5-4c6e-b1d3-62f763883d03 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:17.937029 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7518ff8-8f30-484e-8cbc-0626893864ad admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:17.940604 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1caf7-0f19-436a-9f54-d845ecca0608 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:17.941539 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1caf7-0f19-436a-9f54-d845ecca0608 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:17.942019 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1caf7-0f19-436a-9f54-d845ecca0608 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:17.942444 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1caf7-0f19-436a-9f54-d845ecca0608 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:17.959601 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1017/2022] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:17.959601 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7518ff8-8f30-484e-8cbc-0626893864ad admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:17.962679 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c7518ff8-8f30-484e-8cbc-0626893864ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:17.979188 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7518ff8-8f30-484e-8cbc-0626893864ad admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:17.985426 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b550dd1-ca4f-4f36-89f5-4beb70ec27e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:17.987080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b550dd1-ca4f-4f36-89f5-4beb70ec27e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:17.987413 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b550dd1-ca4f-4f36-89f5-4beb70ec27e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:17.987697 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b550dd1-ca4f-4f36-89f5-4beb70ec27e8 admin admin] PATH_INFO: `/v3/users/87d7fe7d695c4dffbfeffc2edfa1c842` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:17.989321 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b550dd1-ca4f-4f36-89f5-4beb70ec27e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87d7fe7d695c4dffbfeffc2edfa1c842)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:17.992045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b550dd1-ca4f-4f36-89f5-4beb70ec27e8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:18.034465 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1caf7-0f19-436a-9f54-d845ecca0608 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:18.036116 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1caf7-0f19-436a-9f54-d845ecca0608 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:18.049745 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1006/2023] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 16:05:18.050639 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1caf7-0f19-436a-9f54-d845ecca0608 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:18.052400 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0b550dd1-ca4f-4f36-89f5-4beb70ec27e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87d7fe7d695c4dffbfeffc2edfa1c842'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:18.053429 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0b550dd1-ca4f-4f36-89f5-4beb70ec27e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87d7fe7d695c4dffbfeffc2edfa1c842'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:18.054846 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0b550dd1-ca4f-4f36-89f5-4beb70ec27e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87d7fe7d695c4dffbfeffc2edfa1c842'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:18.062977 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0b550dd1-ca4f-4f36-89f5-4beb70ec27e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87d7fe7d695c4dffbfeffc2edfa1c842'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:18.065650 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1caf7-0f19-436a-9f54-d845ecca0608 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:18.071275 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef4eb13-ef14-4c62-a99c-33ee2de7e068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:18.072658 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4eb13-ef14-4c62-a99c-33ee2de7e068 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:18.073034 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4eb13-ef14-4c62-a99c-33ee2de7e068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:18.074271 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4eb13-ef14-4c62-a99c-33ee2de7e068 admin admin] PATH_INFO: `/v3/projects/f66a04ebde8a4e628b700eee4ea2448f/users/95d1419f03104742a2bd3c56a840112e/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:18.075432 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0b550dd1-ca4f-4f36-89f5-4beb70ec27e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87d7fe7d695c4dffbfeffc2edfa1c842'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:18.076410 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef4eb13-ef14-4c62-a99c-33ee2de7e068 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f66a04ebde8a4e628b700eee4ea2448f, user_id=95d1419f03104742a2bd3c56a840112e, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:18.079245 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef4eb13-ef14-4c62-a99c-33ee2de7e068 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:18.100262 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1007/2024] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:18 2022] PUT /identity/v3/projects/f66a04ebde8a4e628b700eee4ea2448f/users/95d1419f03104742a2bd3c56a840112e/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:18.115962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1018/2025] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:17 2022] DELETE /identity/v3/users/87d7fe7d695c4dffbfeffc2edfa1c842 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:18.120329 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90919178-e761-4f23-81a2-4e56bc84af19 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:18.120624 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90919178-e761-4f23-81a2-4e56bc84af19 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:18.120902 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90919178-e761-4f23-81a2-4e56bc84af19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:18.140801 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-90919178-e761-4f23-81a2-4e56bc84af19 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:18.142083 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-90919178-e761-4f23-81a2-4e56bc84af19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:18.244684 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1008/2026] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:18.272568 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b550dd1-ca4f-4f36-89f5-4beb70ec27e8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:18.275439 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0b550dd1-ca4f-4f36-89f5-4beb70ec27e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:18.465794 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b550dd1-ca4f-4f36-89f5-4beb70ec27e8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:18.476428 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54a4fa-3e84-4404-a9a6-036ce07ff72b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:18.477285 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54a4fa-3e84-4404-a9a6-036ce07ff72b service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:18.478189 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54a4fa-3e84-4404-a9a6-036ce07ff72b service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:18.478838 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54a4fa-3e84-4404-a9a6-036ce07ff72b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:18.492544 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec54a4fa-3e84-4404-a9a6-036ce07ff72b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:18.494604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec54a4fa-3e84-4404-a9a6-036ce07ff72b service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:18.508858 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1019/2027] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:18.668634 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90919178-e761-4f23-81a2-4e56bc84af19 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:18.670225 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-90919178-e761-4f23-81a2-4e56bc84af19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:18.690115 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90919178-e761-4f23-81a2-4e56bc84af19 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:18.701412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6782e7-5a3a-4842-b4a3-e399caf6d605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:18.702426 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6782e7-5a3a-4842-b4a3-e399caf6d605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:18.702713 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6782e7-5a3a-4842-b4a3-e399caf6d605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:18.703079 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6782e7-5a3a-4842-b4a3-e399caf6d605 admin admin] PATH_INFO: `/v3/users/aade5ac32498406db53beaf6d87ba3a6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:18.706173 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6782e7-5a3a-4842-b4a3-e399caf6d605 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aade5ac32498406db53beaf6d87ba3a6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:18.711826 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6782e7-5a3a-4842-b4a3-e399caf6d605 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:18.746988 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54a4fa-3e84-4404-a9a6-036ce07ff72b service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:18.749705 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ec54a4fa-3e84-4404-a9a6-036ce07ff72b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:18.778899 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54a4fa-3e84-4404-a9a6-036ce07ff72b service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:18.784069 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde4cac-e793-4122-b71e-fb68bf20f311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:18.785530 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde4cac-e793-4122-b71e-fb68bf20f311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:18.786023 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde4cac-e793-4122-b71e-fb68bf20f311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:18.787160 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde4cac-e793-4122-b71e-fb68bf20f311 admin admin] PATH_INFO: `/v3/users/9f29058d936b41be9273be9559b0269e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:18.789189 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde4cac-e793-4122-b71e-fb68bf20f311 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f29058d936b41be9273be9559b0269e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:18.790858 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde4cac-e793-4122-b71e-fb68bf20f311 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:18.801708 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:18.821165 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7e6782e7-5a3a-4842-b4a3-e399caf6d605 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aade5ac32498406db53beaf6d87ba3a6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:18.822389 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7e6782e7-5a3a-4842-b4a3-e399caf6d605 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aade5ac32498406db53beaf6d87ba3a6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:18.823267 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7e6782e7-5a3a-4842-b4a3-e399caf6d605 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aade5ac32498406db53beaf6d87ba3a6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:18.833238 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7e6782e7-5a3a-4842-b4a3-e399caf6d605 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aade5ac32498406db53beaf6d87ba3a6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:18.845687 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7e6782e7-5a3a-4842-b4a3-e399caf6d605 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aade5ac32498406db53beaf6d87ba3a6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:18.867244 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:18.878211 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1009/2028] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:18 2022] DELETE /identity/v3/users/aade5ac32498406db53beaf6d87ba3a6 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:18.879648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6782e7-5a3a-4842-b4a3-e399caf6d605 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:18.883341 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7e6782e7-5a3a-4842-b4a3-e399caf6d605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:18.886453 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6bde4cac-e793-4122-b71e-fb68bf20f311 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f29058d936b41be9273be9559b0269e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:18.887434 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6bde4cac-e793-4122-b71e-fb68bf20f311 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f29058d936b41be9273be9559b0269e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:18.889394 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6bde4cac-e793-4122-b71e-fb68bf20f311 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f29058d936b41be9273be9559b0269e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:18.911214 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6bde4cac-e793-4122-b71e-fb68bf20f311 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f29058d936b41be9273be9559b0269e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:18.940088 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6bde4cac-e793-4122-b71e-fb68bf20f311 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f29058d936b41be9273be9559b0269e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:18.981543 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6782e7-5a3a-4842-b4a3-e399caf6d605 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:18.992071 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1020/2029] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:18 2022] DELETE /identity/v3/users/9f29058d936b41be9273be9559b0269e => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:18.994275 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142efd5-e195-4674-98ec-0b89886de4e5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:18.994598 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142efd5-e195-4674-98ec-0b89886de4e5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:18.994884 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142efd5-e195-4674-98ec-0b89886de4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:19.002513 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3333566a-2b3a-420d-bf7d-932ef0a40ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:19.003653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333566a-2b3a-420d-bf7d-932ef0a40ae6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:19.004033 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333566a-2b3a-420d-bf7d-932ef0a40ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:19.004499 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333566a-2b3a-420d-bf7d-932ef0a40ae6 admin admin] PATH_INFO: `/v3/users/0160daa137e640faac805671e0fafe38` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:19.013062 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3333566a-2b3a-420d-bf7d-932ef0a40ae6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0160daa137e640faac805671e0fafe38)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:19.015263 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3333566a-2b3a-420d-bf7d-932ef0a40ae6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:19.018251 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a142efd5-e195-4674-98ec-0b89886de4e5 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:19.019729 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a142efd5-e195-4674-98ec-0b89886de4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:19.111285 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1021/2030] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:19.112939 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa23f6-35f4-4c06-a362-76b1ffd0fb44 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:19.113406 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa23f6-35f4-4c06-a362-76b1ffd0fb44 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:19.113834 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa23f6-35f4-4c06-a362-76b1ffd0fb44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:19.119321 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3333566a-2b3a-420d-bf7d-932ef0a40ae6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0160daa137e640faac805671e0fafe38'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:19.120310 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3333566a-2b3a-420d-bf7d-932ef0a40ae6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0160daa137e640faac805671e0fafe38'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:19.121078 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3333566a-2b3a-420d-bf7d-932ef0a40ae6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0160daa137e640faac805671e0fafe38'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:19.138933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2eaa23f6-35f4-4c06-a362-76b1ffd0fb44 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:19.140896 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3333566a-2b3a-420d-bf7d-932ef0a40ae6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0160daa137e640faac805671e0fafe38'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:19.141609 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2eaa23f6-35f4-4c06-a362-76b1ffd0fb44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:19.155501 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1022/2031] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:19.157670 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3333566a-2b3a-420d-bf7d-932ef0a40ae6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0160daa137e640faac805671e0fafe38'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:19.160921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa23f6-35f4-4c06-a362-76b1ffd0fb44 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:19.162797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2eaa23f6-35f4-4c06-a362-76b1ffd0fb44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:19.198777 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1010/2032] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:18 2022] DELETE /identity/v3/users/0160daa137e640faac805671e0fafe38 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:19.202344 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3333566a-2b3a-420d-bf7d-932ef0a40ae6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:19.203926 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3333566a-2b3a-420d-bf7d-932ef0a40ae6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:19.248683 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:19.302646 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3333566a-2b3a-420d-bf7d-932ef0a40ae6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:19.310627 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab1c2f-a64a-49a3-8f48-c39412a07ffd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:19.311822 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab1c2f-a64a-49a3-8f48-c39412a07ffd service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:19.312300 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab1c2f-a64a-49a3-8f48-c39412a07ffd service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:19.312903 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab1c2f-a64a-49a3-8f48-c39412a07ffd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:19.315829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa23f6-35f4-4c06-a362-76b1ffd0fb44 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:19.332015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561aa1c-ca39-4dfd-b3b7-a44aff80f150 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:19.334711 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561aa1c-ca39-4dfd-b3b7-a44aff80f150 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:19.335603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561aa1c-ca39-4dfd-b3b7-a44aff80f150 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:19.336533 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561aa1c-ca39-4dfd-b3b7-a44aff80f150 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:19.340798 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9561aa1c-ca39-4dfd-b3b7-a44aff80f150 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:19.463864 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab1c2f-a64a-49a3-8f48-c39412a07ffd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:19.466165 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab1c2f-a64a-49a3-8f48-c39412a07ffd service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:19.481787 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9561aa1c-ca39-4dfd-b3b7-a44aff80f150 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:19.482359 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1011/2033] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:19.483095 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9561aa1c-ca39-4dfd-b3b7-a44aff80f150 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:19.484436 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2193ee-f8fc-474e-8003-592ca611b701 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:19.485047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2193ee-f8fc-474e-8003-592ca611b701 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:19.485412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2193ee-f8fc-474e-8003-592ca611b701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:19.489385 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:19.496479 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1023/2034] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:19.497419 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561aa1c-ca39-4dfd-b3b7-a44aff80f150 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:19.499325 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9561aa1c-ca39-4dfd-b3b7-a44aff80f150 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:19.506673 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ce2193ee-f8fc-474e-8003-592ca611b701 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:19.508521 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ce2193ee-f8fc-474e-8003-592ca611b701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:19.514865 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561aa1c-ca39-4dfd-b3b7-a44aff80f150 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:19.524731 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1012/2035] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:19.526850 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89a8cb-9767-4c86-98e1-295508f08a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:19.528139 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89a8cb-9767-4c86-98e1-295508f08a10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:19.528792 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89a8cb-9767-4c86-98e1-295508f08a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:19.529270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89a8cb-9767-4c86-98e1-295508f08a10 admin admin] PATH_INFO: `/v3/projects/60f9ebe8843840599f6a084de9539156` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:19.530926 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a89a8cb-9767-4c86-98e1-295508f08a10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60f9ebe8843840599f6a084de9539156)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:19.533143 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a89a8cb-9767-4c86-98e1-295508f08a10 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:19.563381 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2193ee-f8fc-474e-8003-592ca611b701 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:19.588638 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2193ee-f8fc-474e-8003-592ca611b701 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:19.615353 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4a89a8cb-9767-4c86-98e1-295508f08a10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60f9ebe8843840599f6a084de9539156', 'request_id': 'req-4a89a8cb-9767-4c86-98e1-295508f08a10'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:19.632400 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4a89a8cb-9767-4c86-98e1-295508f08a10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60f9ebe8843840599f6a084de9539156', 'request_id': 'req-4a89a8cb-9767-4c86-98e1-295508f08a10'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:19.640026 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4a89a8cb-9767-4c86-98e1-295508f08a10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60f9ebe8843840599f6a084de9539156', 'request_id': 'req-4a89a8cb-9767-4c86-98e1-295508f08a10'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:19.659264 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4a89a8cb-9767-4c86-98e1-295508f08a10 admin admin] The token cache is being invalidate because project 60f9ebe8843840599f6a084de9539156 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:19.660135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4a89a8cb-9767-4c86-98e1-295508f08a10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:19.660564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4a89a8cb-9767-4c86-98e1-295508f08a10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:19.666190 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1024/2036] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:19 2022] DELETE /identity/v3/projects/60f9ebe8843840599f6a084de9539156 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:19.666190 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eface2-3bb9-439b-9646-9088df0ece7d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:19.666190 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eface2-3bb9-439b-9646-9088df0ece7d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:19.666190 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eface2-3bb9-439b-9646-9088df0ece7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:19.691391 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96e190f-8b41-49ee-9dea-aeeea77218aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:19.692606 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e190f-8b41-49ee-9dea-aeeea77218aa service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:19.693085 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e190f-8b41-49ee-9dea-aeeea77218aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:19.694019 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e190f-8b41-49ee-9dea-aeeea77218aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:19.698141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e96e190f-8b41-49ee-9dea-aeeea77218aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:19.705978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c4eface2-3bb9-439b-9646-9088df0ece7d None None] MFA Rules not processed for user `95d1419f03104742a2bd3c56a840112e`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:19.707859 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c4eface2-3bb9-439b-9646-9088df0ece7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:19.816421 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96e190f-8b41-49ee-9dea-aeeea77218aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:19.819195 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96e190f-8b41-49ee-9dea-aeeea77218aa service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:19.823112 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1025/2037] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:19 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:19.824204 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eface2-3bb9-439b-9646-9088df0ece7d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:19.825683 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c4eface2-3bb9-439b-9646-9088df0ece7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:19.841324 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1013/2038] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:19.843677 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96e190f-8b41-49ee-9dea-aeeea77218aa service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:19.846702 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e96e190f-8b41-49ee-9dea-aeeea77218aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:19.849971 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eface2-3bb9-439b-9646-9088df0ece7d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:19.856889 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934712b-0337-425a-b2c1-6d342f64f226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:19.857995 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934712b-0337-425a-b2c1-6d342f64f226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:19.858433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934712b-0337-425a-b2c1-6d342f64f226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:19.858843 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934712b-0337-425a-b2c1-6d342f64f226 admin admin] PATH_INFO: `/v3/users/2dbe3a4500974e57b875b8c10a62d10e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:19.861257 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5934712b-0337-425a-b2c1-6d342f64f226 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dbe3a4500974e57b875b8c10a62d10e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:19.863983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5934712b-0337-425a-b2c1-6d342f64f226 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:19.902446 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:19.939568 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5934712b-0337-425a-b2c1-6d342f64f226 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dbe3a4500974e57b875b8c10a62d10e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:19.942672 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5934712b-0337-425a-b2c1-6d342f64f226 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dbe3a4500974e57b875b8c10a62d10e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:19.943598 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5934712b-0337-425a-b2c1-6d342f64f226 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dbe3a4500974e57b875b8c10a62d10e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:19.955631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96e190f-8b41-49ee-9dea-aeeea77218aa service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:19.960970 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc2c1f-956d-4db3-99ee-5d73a3b62fdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:19.961149 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5934712b-0337-425a-b2c1-6d342f64f226 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dbe3a4500974e57b875b8c10a62d10e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:19.963657 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc2c1f-956d-4db3-99ee-5d73a3b62fdf service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:19.963657 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc2c1f-956d-4db3-99ee-5d73a3b62fdf service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:19.963657 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc2c1f-956d-4db3-99ee-5d73a3b62fdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:19.965698 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3ecc2c1f-956d-4db3-99ee-5d73a3b62fdf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:19.981982 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5934712b-0337-425a-b2c1-6d342f64f226 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dbe3a4500974e57b875b8c10a62d10e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:19.984944 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:20.027088 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1026/2039] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:19 2022] DELETE /identity/v3/users/2dbe3a4500974e57b875b8c10a62d10e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:20.100365 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecc2c1f-956d-4db3-99ee-5d73a3b62fdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:20.102889 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecc2c1f-956d-4db3-99ee-5d73a3b62fdf service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:20.129832 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1014/2040] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:20.425748 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934712b-0337-425a-b2c1-6d342f64f226 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:20.427223 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5934712b-0337-425a-b2c1-6d342f64f226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:20.632367 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934712b-0337-425a-b2c1-6d342f64f226 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:20.648842 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd35dc-9c1a-4013-a934-fb2aa675800a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:20.649775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd35dc-9c1a-4013-a934-fb2aa675800a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:20.650060 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd35dc-9c1a-4013-a934-fb2aa675800a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:20.650335 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd35dc-9c1a-4013-a934-fb2aa675800a admin admin] PATH_INFO: `/v3/projects/328d804f3dfe4aeca17981877380bcde` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:20.652839 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cd35dc-9c1a-4013-a934-fb2aa675800a admin admin] RBAC: Authorizing `identity:delete_project(project_id=328d804f3dfe4aeca17981877380bcde)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:20.655192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cd35dc-9c1a-4013-a934-fb2aa675800a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:20.734596 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-70cd35dc-9c1a-4013-a934-fb2aa675800a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '328d804f3dfe4aeca17981877380bcde', 'request_id': 'req-70cd35dc-9c1a-4013-a934-fb2aa675800a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:20.747248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-70cd35dc-9c1a-4013-a934-fb2aa675800a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '328d804f3dfe4aeca17981877380bcde', 'request_id': 'req-70cd35dc-9c1a-4013-a934-fb2aa675800a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:20.756354 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-70cd35dc-9c1a-4013-a934-fb2aa675800a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '328d804f3dfe4aeca17981877380bcde', 'request_id': 'req-70cd35dc-9c1a-4013-a934-fb2aa675800a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:20.773086 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-70cd35dc-9c1a-4013-a934-fb2aa675800a admin admin] The token cache is being invalidate because project 328d804f3dfe4aeca17981877380bcde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:20.773407 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-70cd35dc-9c1a-4013-a934-fb2aa675800a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:20.774524 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-70cd35dc-9c1a-4013-a934-fb2aa675800a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:20.779122 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1027/2041] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:20 2022] DELETE /identity/v3/projects/328d804f3dfe4aeca17981877380bcde => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:20.783457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc2c1f-956d-4db3-99ee-5d73a3b62fdf service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:20.797036 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc2c1f-956d-4db3-99ee-5d73a3b62fdf service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:20.879897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20d2547-1e8b-4bc2-aa5a-2d3e9d81dbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:20.880939 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d2547-1e8b-4bc2-aa5a-2d3e9d81dbe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:20.881621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d2547-1e8b-4bc2-aa5a-2d3e9d81dbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:20.882025 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d2547-1e8b-4bc2-aa5a-2d3e9d81dbe3 admin admin] PATH_INFO: `/v3/users/2017826094f443ba9e6a62610b6773f8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:20.884764 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20d2547-1e8b-4bc2-aa5a-2d3e9d81dbe3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2017826094f443ba9e6a62610b6773f8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:20.893877 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20d2547-1e8b-4bc2-aa5a-2d3e9d81dbe3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:20.975717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e20d2547-1e8b-4bc2-aa5a-2d3e9d81dbe3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2017826094f443ba9e6a62610b6773f8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:20.976906 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e20d2547-1e8b-4bc2-aa5a-2d3e9d81dbe3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2017826094f443ba9e6a62610b6773f8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:20.977632 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e20d2547-1e8b-4bc2-aa5a-2d3e9d81dbe3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2017826094f443ba9e6a62610b6773f8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:20.987784 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e20d2547-1e8b-4bc2-aa5a-2d3e9d81dbe3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2017826094f443ba9e6a62610b6773f8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:21.000316 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e20d2547-1e8b-4bc2-aa5a-2d3e9d81dbe3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2017826094f443ba9e6a62610b6773f8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:21.024072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd35dc-9c1a-4013-a934-fb2aa675800a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:21.028016 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-70cd35dc-9c1a-4013-a934-fb2aa675800a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:21.055114 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:21.065812 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd35dc-9c1a-4013-a934-fb2aa675800a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:21.071131 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1015/2042] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:20 2022] DELETE /identity/v3/users/2017826094f443ba9e6a62610b6773f8 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:21.117306 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20d2547-1e8b-4bc2-aa5a-2d3e9d81dbe3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:21.119941 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e20d2547-1e8b-4bc2-aa5a-2d3e9d81dbe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:21.239241 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033189ae-cec6-4df4-8777-78ed0b34bb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:21.242927 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033189ae-cec6-4df4-8777-78ed0b34bb56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:21.244562 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033189ae-cec6-4df4-8777-78ed0b34bb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:21.245497 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033189ae-cec6-4df4-8777-78ed0b34bb56 admin admin] PATH_INFO: `/v3/projects/b5bc6434f5f5444db3ad35b3f33c5089` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:21.249540 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033189ae-cec6-4df4-8777-78ed0b34bb56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5bc6434f5f5444db3ad35b3f33c5089)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:21.255780 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033189ae-cec6-4df4-8777-78ed0b34bb56 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:21.287163 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:21.290792 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20d2547-1e8b-4bc2-aa5a-2d3e9d81dbe3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:21.299806 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c060f8d-7c62-4455-8441-cd98e912c23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:21.302465 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c060f8d-7c62-4455-8441-cd98e912c23e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:21.302886 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c060f8d-7c62-4455-8441-cd98e912c23e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:21.303264 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c060f8d-7c62-4455-8441-cd98e912c23e admin admin] PATH_INFO: `/v3/projects/af6be1fe5a1a4a6193888065bc580778` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:21.305037 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c060f8d-7c62-4455-8441-cd98e912c23e admin admin] RBAC: Authorizing `identity:delete_project(project_id=af6be1fe5a1a4a6193888065bc580778)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:21.308725 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c060f8d-7c62-4455-8441-cd98e912c23e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:21.348998 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-033189ae-cec6-4df4-8777-78ed0b34bb56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5bc6434f5f5444db3ad35b3f33c5089', 'request_id': 'req-033189ae-cec6-4df4-8777-78ed0b34bb56'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:21.362126 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-033189ae-cec6-4df4-8777-78ed0b34bb56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5bc6434f5f5444db3ad35b3f33c5089', 'request_id': 'req-033189ae-cec6-4df4-8777-78ed0b34bb56'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:21.371486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-033189ae-cec6-4df4-8777-78ed0b34bb56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5bc6434f5f5444db3ad35b3f33c5089', 'request_id': 'req-033189ae-cec6-4df4-8777-78ed0b34bb56'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:21.388073 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-033189ae-cec6-4df4-8777-78ed0b34bb56 admin admin] The token cache is being invalidate because project b5bc6434f5f5444db3ad35b3f33c5089 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:21.388770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-033189ae-cec6-4df4-8777-78ed0b34bb56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:21.389640 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-033189ae-cec6-4df4-8777-78ed0b34bb56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:21.391810 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1028/2043] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:21 2022] DELETE /identity/v3/projects/b5bc6434f5f5444db3ad35b3f33c5089 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:05:21.392813 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033189ae-cec6-4df4-8777-78ed0b34bb56 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:21.394400 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-033189ae-cec6-4df4-8777-78ed0b34bb56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:21.395078 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0c060f8d-7c62-4455-8441-cd98e912c23e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af6be1fe5a1a4a6193888065bc580778', 'request_id': 'req-0c060f8d-7c62-4455-8441-cd98e912c23e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:21.403211 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0c060f8d-7c62-4455-8441-cd98e912c23e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af6be1fe5a1a4a6193888065bc580778', 'request_id': 'req-0c060f8d-7c62-4455-8441-cd98e912c23e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:21.417438 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0c060f8d-7c62-4455-8441-cd98e912c23e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af6be1fe5a1a4a6193888065bc580778', 'request_id': 'req-0c060f8d-7c62-4455-8441-cd98e912c23e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:21.455917 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0c060f8d-7c62-4455-8441-cd98e912c23e admin admin] The token cache is being invalidate because project af6be1fe5a1a4a6193888065bc580778 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:21.456213 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0c060f8d-7c62-4455-8441-cd98e912c23e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:21.456908 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0c060f8d-7c62-4455-8441-cd98e912c23e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:21.465864 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1016/2044] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:21 2022] DELETE /identity/v3/projects/af6be1fe5a1a4a6193888065bc580778 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:05:21.465864 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c060f8d-7c62-4455-8441-cd98e912c23e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:21.466158 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0c060f8d-7c62-4455-8441-cd98e912c23e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:21.511580 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033189ae-cec6-4df4-8777-78ed0b34bb56 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:21.593992 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c060f8d-7c62-4455-8441-cd98e912c23e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:21.603902 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeafb56-8ccd-4b3f-883c-7a709db4e33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:21.605596 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeafb56-8ccd-4b3f-883c-7a709db4e33d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:21.606013 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeafb56-8ccd-4b3f-883c-7a709db4e33d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:21.606303 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeafb56-8ccd-4b3f-883c-7a709db4e33d admin admin] PATH_INFO: `/v3/users/32386a5e9a804abb880341e256331f9a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:21.607734 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeafb56-8ccd-4b3f-883c-7a709db4e33d admin admin] RBAC: Authorizing `identity:delete_user(user_id=32386a5e9a804abb880341e256331f9a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:21.610141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeafb56-8ccd-4b3f-883c-7a709db4e33d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:21.615740 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8afc89f-586d-4b61-9a2b-3c08bbfcd9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:21.617722 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8afc89f-586d-4b61-9a2b-3c08bbfcd9ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:21.618081 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8afc89f-586d-4b61-9a2b-3c08bbfcd9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:21.619779 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8afc89f-586d-4b61-9a2b-3c08bbfcd9ad admin admin] PATH_INFO: `/v3/projects/d708173f297c44d08f289fbda4c0b051` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:21.622296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8afc89f-586d-4b61-9a2b-3c08bbfcd9ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=d708173f297c44d08f289fbda4c0b051)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:21.624470 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8afc89f-586d-4b61-9a2b-3c08bbfcd9ad admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:21.687102 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:21.716586 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bbeafb56-8ccd-4b3f-883c-7a709db4e33d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32386a5e9a804abb880341e256331f9a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:21.718490 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bbeafb56-8ccd-4b3f-883c-7a709db4e33d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32386a5e9a804abb880341e256331f9a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:21.719457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bbeafb56-8ccd-4b3f-883c-7a709db4e33d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32386a5e9a804abb880341e256331f9a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:21.733960 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e8afc89f-586d-4b61-9a2b-3c08bbfcd9ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd708173f297c44d08f289fbda4c0b051', 'request_id': 'req-e8afc89f-586d-4b61-9a2b-3c08bbfcd9ad'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:21.735701 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bbeafb56-8ccd-4b3f-883c-7a709db4e33d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32386a5e9a804abb880341e256331f9a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:21.737903 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:21.761143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bbeafb56-8ccd-4b3f-883c-7a709db4e33d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32386a5e9a804abb880341e256331f9a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:21.762863 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e8afc89f-586d-4b61-9a2b-3c08bbfcd9ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd708173f297c44d08f289fbda4c0b051', 'request_id': 'req-e8afc89f-586d-4b61-9a2b-3c08bbfcd9ad'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:21.784263 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e8afc89f-586d-4b61-9a2b-3c08bbfcd9ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd708173f297c44d08f289fbda4c0b051', 'request_id': 'req-e8afc89f-586d-4b61-9a2b-3c08bbfcd9ad'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:21.807097 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e8afc89f-586d-4b61-9a2b-3c08bbfcd9ad admin admin] The token cache is being invalidate because project d708173f297c44d08f289fbda4c0b051 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:21.807403 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e8afc89f-586d-4b61-9a2b-3c08bbfcd9ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:21.808276 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e8afc89f-586d-4b61-9a2b-3c08bbfcd9ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:21.815045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1029/2045] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:21 2022] DELETE /identity/v3/projects/d708173f297c44d08f289fbda4c0b051 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:21.815045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd27504-f2fc-4fe4-9bec-cd52cb1b2f5a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:21.815045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd27504-f2fc-4fe4-9bec-cd52cb1b2f5a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:21.815045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd27504-f2fc-4fe4-9bec-cd52cb1b2f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:21.819951 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1017/2046] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:21 2022] DELETE /identity/v3/users/32386a5e9a804abb880341e256331f9a => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:21.821937 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeafb56-8ccd-4b3f-883c-7a709db4e33d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:21.824010 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bbeafb56-8ccd-4b3f-883c-7a709db4e33d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:21.852875 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-efd27504-f2fc-4fe4-9bec-cd52cb1b2f5a None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:21.854488 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-efd27504-f2fc-4fe4-9bec-cd52cb1b2f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:21.973516 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeafb56-8ccd-4b3f-883c-7a709db4e33d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:21.979509 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b9c6c-6000-428a-8253-1bc26059a92e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:21.980569 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1030/2047] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:21.981086 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b9c6c-6000-428a-8253-1bc26059a92e service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:21.981309 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd27504-f2fc-4fe4-9bec-cd52cb1b2f5a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:21.981919 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b9c6c-6000-428a-8253-1bc26059a92e service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:21.982415 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b9c6c-6000-428a-8253-1bc26059a92e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:21.982715 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-efd27504-f2fc-4fe4-9bec-cd52cb1b2f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:21.984732 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-aa3b9c6c-6000-428a-8253-1bc26059a92e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:22.011251 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd27504-f2fc-4fe4-9bec-cd52cb1b2f5a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:22.025836 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51aa7e-95b3-4158-81e4-5ebc38d4c5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:22.026952 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51aa7e-95b3-4158-81e4-5ebc38d4c5bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:22.027075 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51aa7e-95b3-4158-81e4-5ebc38d4c5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:22.027265 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51aa7e-95b3-4158-81e4-5ebc38d4c5bc admin admin] PATH_INFO: `/v3/users/cdd0b902a18c45609044c385fa364f77` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:22.036046 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f51aa7e-95b3-4158-81e4-5ebc38d4c5bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdd0b902a18c45609044c385fa364f77)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:22.038622 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f51aa7e-95b3-4158-81e4-5ebc38d4c5bc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:22.082285 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b9c6c-6000-428a-8253-1bc26059a92e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:22.084409 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b9c6c-6000-428a-8253-1bc26059a92e service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:22.096024 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1018/2048] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:21 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:22.097463 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d89ed4-c968-404a-9709-6800541cb757 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:22.097871 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d89ed4-c968-404a-9709-6800541cb757 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:22.098435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d89ed4-c968-404a-9709-6800541cb757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:22.120067 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-78d89ed4-c968-404a-9709-6800541cb757 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:22.121385 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-78d89ed4-c968-404a-9709-6800541cb757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:22.131383 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1019/2049] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:22.132502 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d89ed4-c968-404a-9709-6800541cb757 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:22.133964 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-78d89ed4-c968-404a-9709-6800541cb757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:22.135977 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0f51aa7e-95b3-4158-81e4-5ebc38d4c5bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdd0b902a18c45609044c385fa364f77'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:22.137337 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0f51aa7e-95b3-4158-81e4-5ebc38d4c5bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdd0b902a18c45609044c385fa364f77'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:22.140148 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0f51aa7e-95b3-4158-81e4-5ebc38d4c5bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdd0b902a18c45609044c385fa364f77'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:22.148303 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d89ed4-c968-404a-9709-6800541cb757 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:22.153925 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492580a7-c994-4092-855f-4160ee160bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:22.155329 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492580a7-c994-4092-855f-4160ee160bce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:22.155605 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492580a7-c994-4092-855f-4160ee160bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:22.155890 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492580a7-c994-4092-855f-4160ee160bce admin admin] PATH_INFO: `/v3/projects/d1438a58f975415987bd91e7e93608ae` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:22.157156 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492580a7-c994-4092-855f-4160ee160bce admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1438a58f975415987bd91e7e93608ae)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:22.158572 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0f51aa7e-95b3-4158-81e4-5ebc38d4c5bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdd0b902a18c45609044c385fa364f77'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:22.159151 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492580a7-c994-4092-855f-4160ee160bce admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:22.178374 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0f51aa7e-95b3-4158-81e4-5ebc38d4c5bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdd0b902a18c45609044c385fa364f77'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:22.220277 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1031/2050] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:21 2022] DELETE /identity/v3/users/cdd0b902a18c45609044c385fa364f77 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:22.221001 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51aa7e-95b3-4158-81e4-5ebc38d4c5bc admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:22.230664 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:22.242205 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0f51aa7e-95b3-4158-81e4-5ebc38d4c5bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:22.256676 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-492580a7-c994-4092-855f-4160ee160bce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1438a58f975415987bd91e7e93608ae', 'request_id': 'req-492580a7-c994-4092-855f-4160ee160bce'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:22.271707 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-492580a7-c994-4092-855f-4160ee160bce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1438a58f975415987bd91e7e93608ae', 'request_id': 'req-492580a7-c994-4092-855f-4160ee160bce'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:22.288059 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-492580a7-c994-4092-855f-4160ee160bce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1438a58f975415987bd91e7e93608ae', 'request_id': 'req-492580a7-c994-4092-855f-4160ee160bce'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:22.332189 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-492580a7-c994-4092-855f-4160ee160bce admin admin] The token cache is being invalidate because project d1438a58f975415987bd91e7e93608ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:22.333037 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-492580a7-c994-4092-855f-4160ee160bce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:22.334162 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-492580a7-c994-4092-855f-4160ee160bce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:22.339058 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1020/2051] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:22 2022] DELETE /identity/v3/projects/d1438a58f975415987bd91e7e93608ae => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:22.344761 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2048fcc8-c0c1-4118-ab32-2e30f594d4a5 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:22.344761 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2048fcc8-c0c1-4118-ab32-2e30f594d4a5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:22.345048 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2048fcc8-c0c1-4118-ab32-2e30f594d4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:22.357343 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:22.376365 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2048fcc8-c0c1-4118-ab32-2e30f594d4a5 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:22.378775 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2048fcc8-c0c1-4118-ab32-2e30f594d4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:22.384537 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51aa7e-95b3-4158-81e4-5ebc38d4c5bc admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:22.400870 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a3e87-498d-411d-953a-3140718af789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:22.401604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a3e87-498d-411d-953a-3140718af789 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:22.401891 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a3e87-498d-411d-953a-3140718af789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:22.402173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a3e87-498d-411d-953a-3140718af789 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:22.402919 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6a3e87-498d-411d-953a-3140718af789 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:05:22.403269 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6a3e87-498d-411d-953a-3140718af789 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:22.405833 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1021/2052] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:05:22.415013 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbacf098-439c-4e8a-9df4-bf309e9caa6e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:22.415699 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbacf098-439c-4e8a-9df4-bf309e9caa6e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:22.416211 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbacf098-439c-4e8a-9df4-bf309e9caa6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:22.418837 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6a3e87-498d-411d-953a-3140718af789 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:22.438244 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1032/2053] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:05:22.443166 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a3e87-498d-411d-953a-3140718af789 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:22.446021 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-dc6a3e87-498d-411d-953a-3140718af789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:22.465475 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-cbacf098-439c-4e8a-9df4-bf309e9caa6e None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:22.468084 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cbacf098-439c-4e8a-9df4-bf309e9caa6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:22.482518 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a3e87-498d-411d-953a-3140718af789 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:22.487575 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1022/2054] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:05:22.490863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbacf098-439c-4e8a-9df4-bf309e9caa6e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:22.514858 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a990a75-1c54-4acf-8c7b-4c4efdec4a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:22.516368 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a990a75-1c54-4acf-8c7b-4c4efdec4a20 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:22.516663 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a990a75-1c54-4acf-8c7b-4c4efdec4a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:22.517088 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a990a75-1c54-4acf-8c7b-4c4efdec4a20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:22.517720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbacf098-439c-4e8a-9df4-bf309e9caa6e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:22.519566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a990a75-1c54-4acf-8c7b-4c4efdec4a20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:22.525035 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a990a75-1c54-4acf-8c7b-4c4efdec4a20 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:22.529885 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18be7ec-88c0-42c4-b4d5-59e0d81e93d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:22.530943 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18be7ec-88c0-42c4-b4d5-59e0d81e93d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:22.531450 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18be7ec-88c0-42c4-b4d5-59e0d81e93d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:22.531936 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18be7ec-88c0-42c4-b4d5-59e0d81e93d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:22.532942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18be7ec-88c0-42c4-b4d5-59e0d81e93d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:05:22.534615 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18be7ec-88c0-42c4-b4d5-59e0d81e93d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:22.539490 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18be7ec-88c0-42c4-b4d5-59e0d81e93d8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:22.556575 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1023/2055] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:05:22.564201 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18be7ec-88c0-42c4-b4d5-59e0d81e93d8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:22.574985 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1033/2056] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:22 2022] POST /identity/v3/projects => generated 387 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:22.583750 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a990a75-1c54-4acf-8c7b-4c4efdec4a20 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:22.595689 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18be7ec-88c0-42c4-b4d5-59e0d81e93d8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:22.695760 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338061c3-26c6-4b7e-9bce-af1cecd4aab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:22.696643 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338061c3-26c6-4b7e-9bce-af1cecd4aab6 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:22.696998 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338061c3-26c6-4b7e-9bce-af1cecd4aab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:22.697317 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338061c3-26c6-4b7e-9bce-af1cecd4aab6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:22.698159 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338061c3-26c6-4b7e-9bce-af1cecd4aab6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:22.700687 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338061c3-26c6-4b7e-9bce-af1cecd4aab6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:22.748551 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a990a75-1c54-4acf-8c7b-4c4efdec4a20 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:22.762164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fded159-f174-44e8-8652-1b3569b48153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:22.763975 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fded159-f174-44e8-8652-1b3569b48153 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:22.764441 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fded159-f174-44e8-8652-1b3569b48153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:22.764831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fded159-f174-44e8-8652-1b3569b48153 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:22.765612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fded159-f174-44e8-8652-1b3569b48153 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:22.767880 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fded159-f174-44e8-8652-1b3569b48153 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:22.790457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1024/2057] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:22 2022] POST /identity/v3/projects => generated 385 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:22.791451 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338061c3-26c6-4b7e-9bce-af1cecd4aab6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:22.793881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-338061c3-26c6-4b7e-9bce-af1cecd4aab6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:22.810321 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1034/2058] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:22 2022] POST /identity/v3/users => generated 294 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:22.811373 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fded159-f174-44e8-8652-1b3569b48153 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:22.814300 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2fded159-f174-44e8-8652-1b3569b48153 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:22.829758 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fded159-f174-44e8-8652-1b3569b48153 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:22.883862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338061c3-26c6-4b7e-9bce-af1cecd4aab6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:22.890568 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1e7c29-93ca-4d4f-af81-c5a6dfe9a287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:22.892166 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e7c29-93ca-4d4f-af81-c5a6dfe9a287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:22.892555 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e7c29-93ca-4d4f-af81-c5a6dfe9a287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:22.892830 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e7c29-93ca-4d4f-af81-c5a6dfe9a287 admin admin] PATH_INFO: `/v3/projects/89ff7c99276c4c24833e67d13daf51d6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:22.894812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1e7c29-93ca-4d4f-af81-c5a6dfe9a287 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89ff7c99276c4c24833e67d13daf51d6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:22.897232 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1e7c29-93ca-4d4f-af81-c5a6dfe9a287 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:22.917310 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551bc5e2-f36f-41fd-b950-2a9b353d0179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:22.918349 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551bc5e2-f36f-41fd-b950-2a9b353d0179 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:22.918740 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551bc5e2-f36f-41fd-b950-2a9b353d0179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:22.919276 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551bc5e2-f36f-41fd-b950-2a9b353d0179 admin admin] PATH_INFO: `/v3/projects/5e8155b725ec4a9e941adbe6390f306f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:22.920592 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551bc5e2-f36f-41fd-b950-2a9b353d0179 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e8155b725ec4a9e941adbe6390f306f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:22.923016 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551bc5e2-f36f-41fd-b950-2a9b353d0179 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:22.950812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:22.969091 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6c1e7c29-93ca-4d4f-af81-c5a6dfe9a287 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89ff7c99276c4c24833e67d13daf51d6', 'request_id': 'req-6c1e7c29-93ca-4d4f-af81-c5a6dfe9a287'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:22.980350 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6c1e7c29-93ca-4d4f-af81-c5a6dfe9a287 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89ff7c99276c4c24833e67d13daf51d6', 'request_id': 'req-6c1e7c29-93ca-4d4f-af81-c5a6dfe9a287'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:22.986431 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-551bc5e2-f36f-41fd-b950-2a9b353d0179 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e8155b725ec4a9e941adbe6390f306f', 'request_id': 'req-551bc5e2-f36f-41fd-b950-2a9b353d0179'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:22.990621 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:22.993393 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6c1e7c29-93ca-4d4f-af81-c5a6dfe9a287 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89ff7c99276c4c24833e67d13daf51d6', 'request_id': 'req-6c1e7c29-93ca-4d4f-af81-c5a6dfe9a287'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:22.996428 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-551bc5e2-f36f-41fd-b950-2a9b353d0179 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e8155b725ec4a9e941adbe6390f306f', 'request_id': 'req-551bc5e2-f36f-41fd-b950-2a9b353d0179'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:23.008286 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-551bc5e2-f36f-41fd-b950-2a9b353d0179 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e8155b725ec4a9e941adbe6390f306f', 'request_id': 'req-551bc5e2-f36f-41fd-b950-2a9b353d0179'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:23.009691 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6c1e7c29-93ca-4d4f-af81-c5a6dfe9a287 admin admin] The token cache is being invalidate because project 89ff7c99276c4c24833e67d13daf51d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:23.010177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6c1e7c29-93ca-4d4f-af81-c5a6dfe9a287 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:23.011798 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6c1e7c29-93ca-4d4f-af81-c5a6dfe9a287 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:23.012618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1025/2059] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:22 2022] DELETE /identity/v3/projects/89ff7c99276c4c24833e67d13daf51d6 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:23.013472 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1e7c29-93ca-4d4f-af81-c5a6dfe9a287 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:23.015035 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6c1e7c29-93ca-4d4f-af81-c5a6dfe9a287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:23.027411 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-551bc5e2-f36f-41fd-b950-2a9b353d0179 admin admin] The token cache is being invalidate because project 5e8155b725ec4a9e941adbe6390f306f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:23.027750 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-551bc5e2-f36f-41fd-b950-2a9b353d0179 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:23.028541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-551bc5e2-f36f-41fd-b950-2a9b353d0179 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:23.030279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1035/2060] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:22 2022] DELETE /identity/v3/projects/5e8155b725ec4a9e941adbe6390f306f => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:23.031212 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551bc5e2-f36f-41fd-b950-2a9b353d0179 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:23.032649 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-551bc5e2-f36f-41fd-b950-2a9b353d0179 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:23.111378 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551bc5e2-f36f-41fd-b950-2a9b353d0179 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:23.116199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1e7c29-93ca-4d4f-af81-c5a6dfe9a287 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:23.117058 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a64a2-696e-4976-aa20-1a210cd6227d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:23.117796 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a64a2-696e-4976-aa20-1a210cd6227d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:23.118181 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a64a2-696e-4976-aa20-1a210cd6227d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:23.118464 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a64a2-696e-4976-aa20-1a210cd6227d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:23.119262 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40a64a2-696e-4976-aa20-1a210cd6227d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:23.121052 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40a64a2-696e-4976-aa20-1a210cd6227d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:23.121889 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6957753b-3743-4442-9096-720dcad56be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:23.122577 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957753b-3743-4442-9096-720dcad56be1 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:23.122870 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957753b-3743-4442-9096-720dcad56be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:23.123159 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957753b-3743-4442-9096-720dcad56be1 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:23.123872 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6957753b-3743-4442-9096-720dcad56be1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:23.125830 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6957753b-3743-4442-9096-720dcad56be1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:23.132296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1036/2061] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:23.133545 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333f1c5-d740-441f-b60d-46057263cdae None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:23.133986 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333f1c5-d740-441f-b60d-46057263cdae None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:23.134265 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333f1c5-d740-441f-b60d-46057263cdae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:23.156613 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1026/2062] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:23 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:23.159663 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162711eb-81f1-41b0-b1e8-20623ca1bbdf None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:23.160506 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162711eb-81f1-41b0-b1e8-20623ca1bbdf None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:23.160926 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162711eb-81f1-41b0-b1e8-20623ca1bbdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:23.164017 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0333f1c5-d740-441f-b60d-46057263cdae None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:23.165267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0333f1c5-d740-441f-b60d-46057263cdae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:23.174566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1037/2063] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:23.175406 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333f1c5-d740-441f-b60d-46057263cdae None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:23.192823 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-162711eb-81f1-41b0-b1e8-20623ca1bbdf None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:23.195185 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333f1c5-d740-441f-b60d-46057263cdae None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:23.195482 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-162711eb-81f1-41b0-b1e8-20623ca1bbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:23.200901 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e61b688-4bd0-47be-bf46-4c9e6742ee19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:23.201626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61b688-4bd0-47be-bf46-4c9e6742ee19 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:23.201933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61b688-4bd0-47be-bf46-4c9e6742ee19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:23.202201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61b688-4bd0-47be-bf46-4c9e6742ee19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:23.202989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e61b688-4bd0-47be-bf46-4c9e6742ee19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:23.211196 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e61b688-4bd0-47be-bf46-4c9e6742ee19 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:23.211399 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1027/2064] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:23.211399 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162711eb-81f1-41b0-b1e8-20623ca1bbdf None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:23.211399 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-162711eb-81f1-41b0-b1e8-20623ca1bbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:23.216542 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1038/2065] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:23.221399 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162711eb-81f1-41b0-b1e8-20623ca1bbdf None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:23.222391 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd337d8e-2a36-4225-a353-068ae80b667e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:23.222752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd337d8e-2a36-4225-a353-068ae80b667e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:23.223357 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd337d8e-2a36-4225-a353-068ae80b667e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:23.226908 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727dcc87-9e2d-42c3-b336-f7551c0a402d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:23.227659 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727dcc87-9e2d-42c3-b336-f7551c0a402d admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:23.228143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727dcc87-9e2d-42c3-b336-f7551c0a402d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:23.228501 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727dcc87-9e2d-42c3-b336-f7551c0a402d admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:23.229316 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727dcc87-9e2d-42c3-b336-f7551c0a402d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:23.231490 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727dcc87-9e2d-42c3-b336-f7551c0a402d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:23.246727 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-bd337d8e-2a36-4225-a353-068ae80b667e None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:23.248601 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bd337d8e-2a36-4225-a353-068ae80b667e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:23.261188 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1039/2066] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:23.263140 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e1e7b-88c4-40ed-9fa5-ec4fce1b310d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:23.263523 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e1e7b-88c4-40ed-9fa5-ec4fce1b310d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:23.264118 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e1e7b-88c4-40ed-9fa5-ec4fce1b310d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:23.270245 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1028/2067] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:23.271295 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727dcc87-9e2d-42c3-b336-f7551c0a402d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:23.284921 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727dcc87-9e2d-42c3-b336-f7551c0a402d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:23.289441 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-683e1e7b-88c4-40ed-9fa5-ec4fce1b310d None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:23.290862 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-683e1e7b-88c4-40ed-9fa5-ec4fce1b310d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:23.292081 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e9988-ed75-4843-a5d0-e63dab0f7b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:23.293712 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e9988-ed75-4843-a5d0-e63dab0f7b77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:23.294030 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e9988-ed75-4843-a5d0-e63dab0f7b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:23.294316 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e9988-ed75-4843-a5d0-e63dab0f7b77 admin admin] PATH_INFO: `/v3/projects/18f6ce66d6834a13b6cacd807445af1a/users/b8ec2f6793df41428a48cf13fe1c4cc4/roles/54195e660fc349978b39d5794765a095` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:23.300992 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1040/2068] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:23.301861 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e1e7b-88c4-40ed-9fa5-ec4fce1b310d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:23.313452 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7e9988-ed75-4843-a5d0-e63dab0f7b77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18f6ce66d6834a13b6cacd807445af1a, user_id=b8ec2f6793df41428a48cf13fe1c4cc4, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:23.313685 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e1e7b-88c4-40ed-9fa5-ec4fce1b310d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:23.316206 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7e9988-ed75-4843-a5d0-e63dab0f7b77 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:23.320524 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa55edb-1631-4054-9470-4caf82abe3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:23.321566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa55edb-1631-4054-9470-4caf82abe3bd admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:23.322091 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa55edb-1631-4054-9470-4caf82abe3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:23.322606 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa55edb-1631-4054-9470-4caf82abe3bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:23.323715 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa55edb-1631-4054-9470-4caf82abe3bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:05:23.324324 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa55edb-1631-4054-9470-4caf82abe3bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:23.327137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa55edb-1631-4054-9470-4caf82abe3bd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:23.340111 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1029/2069] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:23 2022] PUT /identity/v3/projects/18f6ce66d6834a13b6cacd807445af1a/users/b8ec2f6793df41428a48cf13fe1c4cc4/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:23.341151 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e9988-ed75-4843-a5d0-e63dab0f7b77 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:23.342977 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1041/2070] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:05:23.344050 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa55edb-1631-4054-9470-4caf82abe3bd admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:23.357594 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa55edb-1631-4054-9470-4caf82abe3bd admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:23.417062 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eab185-e8c2-4925-8f38-050d52fa401c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:23.417806 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eab185-e8c2-4925-8f38-050d52fa401c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:23.418094 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eab185-e8c2-4925-8f38-050d52fa401c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:23.418369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eab185-e8c2-4925-8f38-050d52fa401c admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:23.419113 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eab185-e8c2-4925-8f38-050d52fa401c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:05:23.419420 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eab185-e8c2-4925-8f38-050d52fa401c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:23.421130 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eab185-e8c2-4925-8f38-050d52fa401c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:23.433718 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1042/2071] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:05:23.434653 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eab185-e8c2-4925-8f38-050d52fa401c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:23.443252 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e9988-ed75-4843-a5d0-e63dab0f7b77 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:23.445972 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eab185-e8c2-4925-8f38-050d52fa401c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:23.450517 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b27f3-3bf9-4c5d-837a-b993932a30d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:23.451540 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b27f3-3bf9-4c5d-837a-b993932a30d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:23.451989 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b27f3-3bf9-4c5d-837a-b993932a30d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:23.452107 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0727eb3-21b2-40f8-b913-964bbd38e672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:23.452312 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b27f3-3bf9-4c5d-837a-b993932a30d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:23.452719 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0727eb3-21b2-40f8-b913-964bbd38e672 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:23.453074 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0727eb3-21b2-40f8-b913-964bbd38e672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:23.453242 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424b27f3-3bf9-4c5d-837a-b993932a30d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:23.453330 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0727eb3-21b2-40f8-b913-964bbd38e672 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:23.454065 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0727eb3-21b2-40f8-b913-964bbd38e672 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:23.455073 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424b27f3-3bf9-4c5d-837a-b993932a30d0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:23.456278 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0727eb3-21b2-40f8-b913-964bbd38e672 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:23.471165 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1030/2072] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:23.471165 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6253a0-f7d0-4c57-9235-9fb865a8cedb None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:23.471165 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6253a0-f7d0-4c57-9235-9fb865a8cedb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:23.471511 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6253a0-f7d0-4c57-9235-9fb865a8cedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:23.488097 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1043/2073] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:23 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:23.489220 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0727eb3-21b2-40f8-b913-964bbd38e672 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:23.498980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:23.501955 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:23.514673 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5a6253a0-f7d0-4c57-9235-9fb865a8cedb None None] MFA Rules not processed for user `b8ec2f6793df41428a48cf13fe1c4cc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:23.516680 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5a6253a0-f7d0-4c57-9235-9fb865a8cedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:23.571901 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0727eb3-21b2-40f8-b913-964bbd38e672 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:23.577329 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0214c-bce6-44bd-9d0f-f7f0d4fd4585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:23.578292 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0214c-bce6-44bd-9d0f-f7f0d4fd4585 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:23.578749 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0214c-bce6-44bd-9d0f-f7f0d4fd4585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:23.579271 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0214c-bce6-44bd-9d0f-f7f0d4fd4585 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:23.580187 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb0214c-bce6-44bd-9d0f-f7f0d4fd4585 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:23.582349 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb0214c-bce6-44bd-9d0f-f7f0d4fd4585 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:23.596997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1031/2074] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:23.598359 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6253a0-f7d0-4c57-9235-9fb865a8cedb None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:23.609979 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6253a0-f7d0-4c57-9235-9fb865a8cedb None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:23.614716 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1251418-f9c7-45f4-8cd9-a30c4126045c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:23.615692 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1251418-f9c7-45f4-8cd9-a30c4126045c admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:23.616572 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1251418-f9c7-45f4-8cd9-a30c4126045c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:23.616936 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1251418-f9c7-45f4-8cd9-a30c4126045c admin admin] PATH_INFO: `/v3/projects/5050d95145f94d63bbe3a5970907bbe7/users/afc50f7c8b444a61a3a43b2aa8ea540e/roles/54195e660fc349978b39d5794765a095` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:23.623862 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1044/2075] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:23 2022] POST /identity/v3/projects => generated 371 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:23.624828 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0214c-bce6-44bd-9d0f-f7f0d4fd4585 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:23.642415 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1251418-f9c7-45f4-8cd9-a30c4126045c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5050d95145f94d63bbe3a5970907bbe7, user_id=afc50f7c8b444a61a3a43b2aa8ea540e, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:23.645241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1251418-f9c7-45f4-8cd9-a30c4126045c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:23.662384 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1032/2076] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:23 2022] PUT /identity/v3/projects/5050d95145f94d63bbe3a5970907bbe7/users/afc50f7c8b444a61a3a43b2aa8ea540e/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:23.663462 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1251418-f9c7-45f4-8cd9-a30c4126045c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:23.726923 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0214c-bce6-44bd-9d0f-f7f0d4fd4585 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:23.747104 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1251418-f9c7-45f4-8cd9-a30c4126045c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:23.751952 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e55a775-fee6-402e-8b6b-e7ae68c7711f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:23.752666 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55a775-fee6-402e-8b6b-e7ae68c7711f admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:23.753059 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55a775-fee6-402e-8b6b-e7ae68c7711f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:23.753265 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55a775-fee6-402e-8b6b-e7ae68c7711f admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:23.754009 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e55a775-fee6-402e-8b6b-e7ae68c7711f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:23.756104 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e55a775-fee6-402e-8b6b-e7ae68c7711f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:23.808954 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e8066-559b-4f7f-b37a-77b38be1bd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:23.810746 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e8066-559b-4f7f-b37a-77b38be1bd2a admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:23.811134 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e8066-559b-4f7f-b37a-77b38be1bd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:23.811503 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e8066-559b-4f7f-b37a-77b38be1bd2a admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:23.812974 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e8066-559b-4f7f-b37a-77b38be1bd2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:23.814636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1033/2077] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:23 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:23.815789 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e8066-559b-4f7f-b37a-77b38be1bd2a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:23.818350 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e55a775-fee6-402e-8b6b-e7ae68c7711f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:23.867342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1045/2078] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:23 2022] POST /identity/v3/users => generated 293 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:23.869604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa237676-c69d-4d69-8a0c-c54b63c6e242 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:23.869997 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa237676-c69d-4d69-8a0c-c54b63c6e242 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:23.870527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa237676-c69d-4d69-8a0c-c54b63c6e242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:23.919336 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e55a775-fee6-402e-8b6b-e7ae68c7711f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:23.921311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fa237676-c69d-4d69-8a0c-c54b63c6e242 None None] MFA Rules not processed for user `afc50f7c8b444a61a3a43b2aa8ea540e`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:23.922375 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fa237676-c69d-4d69-8a0c-c54b63c6e242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:23.925689 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448d6e3-4d46-4bb0-b003-3ab7fd8d4bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:23.926509 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448d6e3-4d46-4bb0-b003-3ab7fd8d4bbc admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:23.926860 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448d6e3-4d46-4bb0-b003-3ab7fd8d4bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:23.927293 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448d6e3-4d46-4bb0-b003-3ab7fd8d4bbc admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:23.928077 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c448d6e3-4d46-4bb0-b003-3ab7fd8d4bbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:23.931780 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c448d6e3-4d46-4bb0-b003-3ab7fd8d4bbc admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:23.939734 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:23.948482 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:23.965795 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1034/2079] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:23 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:23.966706 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448d6e3-4d46-4bb0-b003-3ab7fd8d4bbc admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:23.986675 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448d6e3-4d46-4bb0-b003-3ab7fd8d4bbc admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:23.991808 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d55efda-185c-46c6-b0bd-6fcf7552a4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:23.992617 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55efda-185c-46c6-b0bd-6fcf7552a4b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:23.992786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55efda-185c-46c6-b0bd-6fcf7552a4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:23.993088 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55efda-185c-46c6-b0bd-6fcf7552a4b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:23.993842 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d55efda-185c-46c6-b0bd-6fcf7552a4b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:23.995788 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d55efda-185c-46c6-b0bd-6fcf7552a4b0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:24.025996 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1035/2080] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:23 2022] POST /identity/v3/users => generated 293 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:24.025996 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d55efda-185c-46c6-b0bd-6fcf7552a4b0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:24.030333 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1046/2081] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:24.031113 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa237676-c69d-4d69-8a0c-c54b63c6e242 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:24.039901 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d55efda-185c-46c6-b0bd-6fcf7552a4b0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:24.049660 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91750b-7220-4fab-a968-256c41042444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:24.050535 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91750b-7220-4fab-a968-256c41042444 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:24.050895 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91750b-7220-4fab-a968-256c41042444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:24.051561 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91750b-7220-4fab-a968-256c41042444 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:24.052532 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa237676-c69d-4d69-8a0c-c54b63c6e242 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:24.052655 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91750b-7220-4fab-a968-256c41042444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:24.054938 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91750b-7220-4fab-a968-256c41042444 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:24.059935 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68802d42-7938-447d-9b40-25a07a0dfd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:24.061136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68802d42-7938-447d-9b40-25a07a0dfd28 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:24.062337 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68802d42-7938-447d-9b40-25a07a0dfd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:24.062808 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68802d42-7938-447d-9b40-25a07a0dfd28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:24.063950 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68802d42-7938-447d-9b40-25a07a0dfd28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:24.066041 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68802d42-7938-447d-9b40-25a07a0dfd28 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:24.068195 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1036/2082] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:24.069161 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91750b-7220-4fab-a968-256c41042444 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:24.082597 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1047/2083] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:24.082924 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91750b-7220-4fab-a968-256c41042444 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:24.086547 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68802d42-7938-447d-9b40-25a07a0dfd28 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:24.087420 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ed5e4-653b-47d4-8fce-659590a0f8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:24.088278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ed5e4-653b-47d4-8fce-659590a0f8c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:24.088660 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ed5e4-653b-47d4-8fce-659590a0f8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:24.089036 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ed5e4-653b-47d4-8fce-659590a0f8c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:24.090638 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ed5e4-653b-47d4-8fce-659590a0f8c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:24.092885 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ed5e4-653b-47d4-8fce-659590a0f8c0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:24.100138 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68802d42-7938-447d-9b40-25a07a0dfd28 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:24.106918 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9716a32-a5d3-4649-9ca9-c4a0fabf94ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:24.107988 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9716a32-a5d3-4649-9ca9-c4a0fabf94ed admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:24.111234 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1037/2084] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:24.111234 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ed5e4-653b-47d4-8fce-659590a0f8c0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:24.111757 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9716a32-a5d3-4649-9ca9-c4a0fabf94ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:24.112271 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9716a32-a5d3-4649-9ca9-c4a0fabf94ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:24.113171 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9716a32-a5d3-4649-9ca9-c4a0fabf94ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:24.115335 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9716a32-a5d3-4649-9ca9-c4a0fabf94ed admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:24.121985 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ed5e4-653b-47d4-8fce-659590a0f8c0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:24.128049 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a39b86-9f1f-408f-9abb-d07137a0b52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:24.129546 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a39b86-9f1f-408f-9abb-d07137a0b52c admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:24.129934 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a39b86-9f1f-408f-9abb-d07137a0b52c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:24.130277 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a39b86-9f1f-408f-9abb-d07137a0b52c admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:24.131087 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a39b86-9f1f-408f-9abb-d07137a0b52c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:24.132966 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a39b86-9f1f-408f-9abb-d07137a0b52c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:24.145934 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1038/2085] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:24.146766 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a39b86-9f1f-408f-9abb-d07137a0b52c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:24.154884 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1048/2086] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:24 2022] POST /identity/v3/projects => generated 383 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:24.156006 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9716a32-a5d3-4649-9ca9-c4a0fabf94ed admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:24.158280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a39b86-9f1f-408f-9abb-d07137a0b52c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:24.221597 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d3f13-0f45-4503-8156-d637c9c7fe65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:24.222566 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d3f13-0f45-4503-8156-d637c9c7fe65 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:24.222934 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d3f13-0f45-4503-8156-d637c9c7fe65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:24.223843 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d3f13-0f45-4503-8156-d637c9c7fe65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:24.224884 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09d3f13-0f45-4503-8156-d637c9c7fe65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:24.228208 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09d3f13-0f45-4503-8156-d637c9c7fe65 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:24.231161 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9716a32-a5d3-4649-9ca9-c4a0fabf94ed admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:24.239643 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1039/2087] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:24.240576 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d3f13-0f45-4503-8156-d637c9c7fe65 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:24.243003 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915613e6-c9a9-4212-a34a-bb8d2432aab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:24.244382 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915613e6-c9a9-4212-a34a-bb8d2432aab7 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:24.244788 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915613e6-c9a9-4212-a34a-bb8d2432aab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:24.245567 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915613e6-c9a9-4212-a34a-bb8d2432aab7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:24.246386 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915613e6-c9a9-4212-a34a-bb8d2432aab7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:24.251291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915613e6-c9a9-4212-a34a-bb8d2432aab7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:24.251933 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d3f13-0f45-4503-8156-d637c9c7fe65 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:24.258083 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0145b-be60-42c5-bc03-91b8446de789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:24.259089 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0145b-be60-42c5-bc03-91b8446de789 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:24.259442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0145b-be60-42c5-bc03-91b8446de789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:24.259799 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0145b-be60-42c5-bc03-91b8446de789 admin admin] PATH_INFO: `/v3/projects/55ae1f94a3bb4f4abe4ab4314030db4f/users/dc454778d4264587ac92146569a66ebd/roles/54195e660fc349978b39d5794765a095` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:24.265266 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1049/2088] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:24.265947 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915613e6-c9a9-4212-a34a-bb8d2432aab7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:24.275996 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915613e6-c9a9-4212-a34a-bb8d2432aab7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:24.276208 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0145b-be60-42c5-bc03-91b8446de789 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55ae1f94a3bb4f4abe4ab4314030db4f, user_id=dc454778d4264587ac92146569a66ebd, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:24.279153 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0145b-be60-42c5-bc03-91b8446de789 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:24.280684 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e455f-96a2-4c8c-beef-94fd8bda6f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:24.281338 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e455f-96a2-4c8c-beef-94fd8bda6f44 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:24.281671 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e455f-96a2-4c8c-beef-94fd8bda6f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:24.281962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e455f-96a2-4c8c-beef-94fd8bda6f44 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:24.282717 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9e455f-96a2-4c8c-beef-94fd8bda6f44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:24.285084 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9e455f-96a2-4c8c-beef-94fd8bda6f44 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:24.295712 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1040/2089] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:24 2022] PUT /identity/v3/projects/55ae1f94a3bb4f4abe4ab4314030db4f/users/dc454778d4264587ac92146569a66ebd/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:24.296555 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0145b-be60-42c5-bc03-91b8446de789 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:24.311141 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1050/2090] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:24 2022] POST /identity/v3/users => generated 298 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:24.312016 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e455f-96a2-4c8c-beef-94fd8bda6f44 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:24.321924 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e455f-96a2-4c8c-beef-94fd8bda6f44 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:24.381708 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0145b-be60-42c5-bc03-91b8446de789 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:24.386639 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af065b6f-29bc-4c5f-a808-835d68c09d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:24.387859 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af065b6f-29bc-4c5f-a808-835d68c09d8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:24.388149 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af065b6f-29bc-4c5f-a808-835d68c09d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:24.388424 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af065b6f-29bc-4c5f-a808-835d68c09d8a admin admin] PATH_INFO: `/v3/projects/7e8fe26b25d44c89a9861a5b8e17e395/users/ad718e6456d542d7a2d99c549909b4c3/roles/54195e660fc349978b39d5794765a095` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:24.395526 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e841c-ca0e-48a5-ac34-65197b02ccda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:24.396479 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e841c-ca0e-48a5-ac34-65197b02ccda admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:24.396767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e841c-ca0e-48a5-ac34-65197b02ccda admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:24.397038 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e841c-ca0e-48a5-ac34-65197b02ccda admin admin] PATH_INFO: `/v3/projects/8ecda84209174d5c884f7a6417449d63/users/daf943301b734d17ab6c4521271a3de5/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:24.404569 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af065b6f-29bc-4c5f-a808-835d68c09d8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e8fe26b25d44c89a9861a5b8e17e395, user_id=ad718e6456d542d7a2d99c549909b4c3, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:24.407300 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af065b6f-29bc-4c5f-a808-835d68c09d8a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:24.409937 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:24.410553 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:24.415599 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2e841c-ca0e-48a5-ac34-65197b02ccda admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ecda84209174d5c884f7a6417449d63, user_id=daf943301b734d17ab6c4521271a3de5, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:24.419161 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2e841c-ca0e-48a5-ac34-65197b02ccda admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:24.424935 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1041/2091] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:24 2022] PUT /identity/v3/projects/7e8fe26b25d44c89a9861a5b8e17e395/users/ad718e6456d542d7a2d99c549909b4c3/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:24.426468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afb855-8b24-42f2-bad6-cc65387e1fa0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:24.426803 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afb855-8b24-42f2-bad6-cc65387e1fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:24.427071 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afb855-8b24-42f2-bad6-cc65387e1fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:24.435463 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1051/2092] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:24 2022] PUT /identity/v3/projects/8ecda84209174d5c884f7a6417449d63/users/daf943301b734d17ab6c4521271a3de5/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:24.436355 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e841c-ca0e-48a5-ac34-65197b02ccda admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:24.453355 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-57afb855-8b24-42f2-bad6-cc65387e1fa0 None None] MFA Rules not processed for user `dc454778d4264587ac92146569a66ebd`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:24.454615 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-57afb855-8b24-42f2-bad6-cc65387e1fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:24.499374 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e841c-ca0e-48a5-ac34-65197b02ccda admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:24.504210 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d274a-d0e6-4537-8095-dbb53219904e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:24.504910 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d274a-d0e6-4537-8095-dbb53219904e admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:24.505207 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d274a-d0e6-4537-8095-dbb53219904e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:24.505480 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d274a-d0e6-4537-8095-dbb53219904e admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:24.506251 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d274a-d0e6-4537-8095-dbb53219904e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:24.508083 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d274a-d0e6-4537-8095-dbb53219904e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:24.521282 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1052/2093] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:24.521282 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4bdacb-01eb-4529-bacd-9fdfcd9b373f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:24.521282 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4bdacb-01eb-4529-bacd-9fdfcd9b373f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:24.521282 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4bdacb-01eb-4529-bacd-9fdfcd9b373f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:24.522874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1042/2094] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:24.522874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c242a19-9fbe-4329-9d8e-880c0a6827db None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:24.522874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c242a19-9fbe-4329-9d8e-880c0a6827db None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:24.523092 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c242a19-9fbe-4329-9d8e-880c0a6827db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:24.544491 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4f4bdacb-01eb-4529-bacd-9fdfcd9b373f None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:24.545756 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4f4bdacb-01eb-4529-bacd-9fdfcd9b373f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:24.550036 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0c242a19-9fbe-4329-9d8e-880c0a6827db None None] MFA Rules not processed for user `daf943301b734d17ab6c4521271a3de5`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:24.551241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0c242a19-9fbe-4329-9d8e-880c0a6827db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:24.554239 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1053/2095] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:24.557871 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4bdacb-01eb-4529-bacd-9fdfcd9b373f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:24.567208 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4bdacb-01eb-4529-bacd-9fdfcd9b373f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:24.572600 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf93dc-c167-4b4d-87ab-02da8750c94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:24.573534 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf93dc-c167-4b4d-87ab-02da8750c94f admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:24.573872 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf93dc-c167-4b4d-87ab-02da8750c94f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:24.574152 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf93dc-c167-4b4d-87ab-02da8750c94f admin admin] PATH_INFO: `/v3/projects/88f6ce0aa585487a859cb31986a80fdb/users/9316a2fa67fc4452ad2809ed43bbf585/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:24.588834 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf93dc-c167-4b4d-87ab-02da8750c94f admin admin] RBAC: Authorizing `identity:create_grant(project_id=88f6ce0aa585487a859cb31986a80fdb, user_id=9316a2fa67fc4452ad2809ed43bbf585, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:24.591554 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf93dc-c167-4b4d-87ab-02da8750c94f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:24.605881 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1054/2096] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:24 2022] PUT /identity/v3/projects/88f6ce0aa585487a859cb31986a80fdb/users/9316a2fa67fc4452ad2809ed43bbf585/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:24.606908 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf93dc-c167-4b4d-87ab-02da8750c94f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:24.610378 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-efaf93dc-c167-4b4d-87ab-02da8750c94f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:24.617018 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1043/2097] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:24.617885 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c242a19-9fbe-4329-9d8e-880c0a6827db None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:24.621272 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0c242a19-9fbe-4329-9d8e-880c0a6827db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:24.673341 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf93dc-c167-4b4d-87ab-02da8750c94f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:24.677285 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43630a8a-a4d3-4fe9-983e-cd593a634727 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:24.677969 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43630a8a-a4d3-4fe9-983e-cd593a634727 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:24.678252 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43630a8a-a4d3-4fe9-983e-cd593a634727 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:24.678848 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43630a8a-a4d3-4fe9-983e-cd593a634727 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:24.680908 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c242a19-9fbe-4329-9d8e-880c0a6827db None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:24.708312 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:24.735275 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d48f7-19dd-45c6-8a60-dcaabc094239 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:24.735933 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d48f7-19dd-45c6-8a60-dcaabc094239 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:24.736220 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d48f7-19dd-45c6-8a60-dcaabc094239 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:24.736500 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d48f7-19dd-45c6-8a60-dcaabc094239 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:24.737976 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43630a8a-a4d3-4fe9-983e-cd593a634727 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:24.739512 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43630a8a-a4d3-4fe9-983e-cd593a634727 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:24.748822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1055/2098] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:24.749830 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43630a8a-a4d3-4fe9-983e-cd593a634727 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:24.758482 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:24.801009 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34d48f7-19dd-45c6-8a60-dcaabc094239 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:24.802466 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34d48f7-19dd-45c6-8a60-dcaabc094239 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:24.812202 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1044/2099] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:24.813008 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d48f7-19dd-45c6-8a60-dcaabc094239 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:24.818379 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43630a8a-a4d3-4fe9-983e-cd593a634727 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:24.825316 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443481b-b40a-4447-8cdc-1fa515c637f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:24.825483 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d48f7-19dd-45c6-8a60-dcaabc094239 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:24.826122 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443481b-b40a-4447-8cdc-1fa515c637f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:24.826416 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443481b-b40a-4447-8cdc-1fa515c637f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:24.826695 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443481b-b40a-4447-8cdc-1fa515c637f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:24.827433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d443481b-b40a-4447-8cdc-1fa515c637f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:24.829113 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d443481b-b40a-4447-8cdc-1fa515c637f5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:24.832716 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0182e-f81f-496c-8dd3-2d467c8ff56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:24.833439 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0182e-f81f-496c-8dd3-2d467c8ff56e admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:24.833853 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0182e-f81f-496c-8dd3-2d467c8ff56e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:24.834123 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0182e-f81f-496c-8dd3-2d467c8ff56e admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:24.835280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b0182e-f81f-496c-8dd3-2d467c8ff56e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:24.837299 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b0182e-f81f-496c-8dd3-2d467c8ff56e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:24.841200 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1056/2100] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:24.842271 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443481b-b40a-4447-8cdc-1fa515c637f5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:24.853563 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443481b-b40a-4447-8cdc-1fa515c637f5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:24.859072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6159ab01-3611-4349-b9d2-f501a20e3dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:24.860256 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159ab01-3611-4349-b9d2-f501a20e3dc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:24.860706 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159ab01-3611-4349-b9d2-f501a20e3dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:24.861060 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159ab01-3611-4349-b9d2-f501a20e3dc0 admin admin] PATH_INFO: `/v3/users/dc454778d4264587ac92146569a66ebd` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:24.862598 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6159ab01-3611-4349-b9d2-f501a20e3dc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc454778d4264587ac92146569a66ebd)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:24.864619 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6159ab01-3611-4349-b9d2-f501a20e3dc0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:24.871689 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1045/2101] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:24 2022] POST /identity/v3/projects => generated 387 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:24.872992 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0182e-f81f-496c-8dd3-2d467c8ff56e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:24.925881 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6159ab01-3611-4349-b9d2-f501a20e3dc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc454778d4264587ac92146569a66ebd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:24.926469 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6159ab01-3611-4349-b9d2-f501a20e3dc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc454778d4264587ac92146569a66ebd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:24.927453 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6159ab01-3611-4349-b9d2-f501a20e3dc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc454778d4264587ac92146569a66ebd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:24.935426 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6159ab01-3611-4349-b9d2-f501a20e3dc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc454778d4264587ac92146569a66ebd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:24.942932 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6159ab01-3611-4349-b9d2-f501a20e3dc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc454778d4264587ac92146569a66ebd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:24.955541 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0182e-f81f-496c-8dd3-2d467c8ff56e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:24.961092 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1574627-a106-4f69-9f00-1d8e576ae5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:24.962038 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1574627-a106-4f69-9f00-1d8e576ae5a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:24.962329 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1574627-a106-4f69-9f00-1d8e576ae5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:24.962612 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1574627-a106-4f69-9f00-1d8e576ae5a1 admin admin] PATH_INFO: `/v3/projects/88f6ce0aa585487a859cb31986a80fdb/users/9316a2fa67fc4452ad2809ed43bbf585/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:24.965974 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1574627-a106-4f69-9f00-1d8e576ae5a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88f6ce0aa585487a859cb31986a80fdb, user_id=9316a2fa67fc4452ad2809ed43bbf585, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:24.968628 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1057/2102] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:24 2022] DELETE /identity/v3/users/dc454778d4264587ac92146569a66ebd => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:24.968774 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1574627-a106-4f69-9f00-1d8e576ae5a1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:24.969329 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6159ab01-3611-4349-b9d2-f501a20e3dc0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:24.974278 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6159ab01-3611-4349-b9d2-f501a20e3dc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:24.986347 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1046/2103] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:24 2022] PUT /identity/v3/projects/88f6ce0aa585487a859cb31986a80fdb/users/9316a2fa67fc4452ad2809ed43bbf585/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:24.986347 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701a05e-f09b-418b-8316-dc0161b70e78 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:24.986347 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701a05e-f09b-418b-8316-dc0161b70e78 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:24.986347 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701a05e-f09b-418b-8316-dc0161b70e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:25.007159 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b701a05e-f09b-418b-8316-dc0161b70e78 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:25.008825 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b701a05e-f09b-418b-8316-dc0161b70e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:25.076070 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:25.079011 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6159ab01-3611-4349-b9d2-f501a20e3dc0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:25.105044 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1047/2104] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:25.106563 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8aaf1-3bd0-4fa6-a5cf-7ef0d60cc5e3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:25.106858 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8aaf1-3bd0-4fa6-a5cf-7ef0d60cc5e3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:25.107141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8aaf1-3bd0-4fa6-a5cf-7ef0d60cc5e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:25.113869 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:25.152125 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5ea8aaf1-3bd0-4fa6-a5cf-7ef0d60cc5e3 None None] MFA Rules not processed for user `9316a2fa67fc4452ad2809ed43bbf585`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:25.153440 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5ea8aaf1-3bd0-4fa6-a5cf-7ef0d60cc5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:25.177317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f16930-48e2-406b-bf62-716d74fc173c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:25.178117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f16930-48e2-406b-bf62-716d74fc173c admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:25.178804 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f16930-48e2-406b-bf62-716d74fc173c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:25.179350 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f16930-48e2-406b-bf62-716d74fc173c admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:25.180329 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f16930-48e2-406b-bf62-716d74fc173c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:25.182444 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f16930-48e2-406b-bf62-716d74fc173c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:25.221356 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1058/2105] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:24 2022] POST /identity/v3/users => generated 300 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:25.222535 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f16930-48e2-406b-bf62-716d74fc173c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:25.227092 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-52f16930-48e2-406b-bf62-716d74fc173c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:25.300100 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1048/2106] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:25 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:25.300100 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea8aaf1-3bd0-4fa6-a5cf-7ef0d60cc5e3 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:25.327140 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f16930-48e2-406b-bf62-716d74fc173c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:25.328035 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea8aaf1-3bd0-4fa6-a5cf-7ef0d60cc5e3 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:25.333168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf725b1d-996c-4f7d-8a44-962d25e537e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:25.334074 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf725b1d-996c-4f7d-8a44-962d25e537e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:25.334562 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf725b1d-996c-4f7d-8a44-962d25e537e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:25.335015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf725b1d-996c-4f7d-8a44-962d25e537e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:25.342508 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fcc7c0-57d4-4e67-a459-f4f74d690f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:25.343447 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fcc7c0-57d4-4e67-a459-f4f74d690f00 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:25.344202 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fcc7c0-57d4-4e67-a459-f4f74d690f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:25.344464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fcc7c0-57d4-4e67-a459-f4f74d690f00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:25.345414 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fcc7c0-57d4-4e67-a459-f4f74d690f00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:25.349398 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf725b1d-996c-4f7d-8a44-962d25e537e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:25.350637 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fcc7c0-57d4-4e67-a459-f4f74d690f00 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:25.351709 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf725b1d-996c-4f7d-8a44-962d25e537e9 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:25.383014 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1059/2107] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:25.383583 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1049/2108] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:25.388111 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fcc7c0-57d4-4e67-a459-f4f74d690f00 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:25.388446 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf725b1d-996c-4f7d-8a44-962d25e537e9 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:25.391024 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bf725b1d-996c-4f7d-8a44-962d25e537e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:25.411129 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fcc7c0-57d4-4e67-a459-f4f74d690f00 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:25.417177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc78643-428a-4980-a832-a718da9dde39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:25.418711 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc78643-428a-4980-a832-a718da9dde39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:25.419368 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc78643-428a-4980-a832-a718da9dde39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:25.419665 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc78643-428a-4980-a832-a718da9dde39 admin admin] PATH_INFO: `/v3/projects/286b999288bf427bb09c3a2c2ab418a7/users/b6aafb34086b4f8a95a7f52aea356f12/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:25.446695 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc78643-428a-4980-a832-a718da9dde39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=286b999288bf427bb09c3a2c2ab418a7, user_id=b6aafb34086b4f8a95a7f52aea356f12, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:25.450474 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc78643-428a-4980-a832-a718da9dde39 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:25.473694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1050/2109] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:25 2022] PUT /identity/v3/projects/286b999288bf427bb09c3a2c2ab418a7/users/b6aafb34086b4f8a95a7f52aea356f12/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:25.474108 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf725b1d-996c-4f7d-8a44-962d25e537e9 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:25.474614 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc78643-428a-4980-a832-a718da9dde39 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:25.534797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0af6c4-a4b0-4159-bbe2-afc7804e9417 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:25.534797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0af6c4-a4b0-4159-bbe2-afc7804e9417 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:25.534797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0af6c4-a4b0-4159-bbe2-afc7804e9417 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:25.535228 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0af6c4-a4b0-4159-bbe2-afc7804e9417 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:25.537240 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0a0af6c4-a4b0-4159-bbe2-afc7804e9417 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:25.574563 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:25.601778 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc78643-428a-4980-a832-a718da9dde39 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:25.608326 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d9ee11-3ff3-4362-9c99-22ba2b0fb8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:25.609174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d9ee11-3ff3-4362-9c99-22ba2b0fb8cb admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:25.609502 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d9ee11-3ff3-4362-9c99-22ba2b0fb8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:25.609821 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d9ee11-3ff3-4362-9c99-22ba2b0fb8cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:25.610808 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d9ee11-3ff3-4362-9c99-22ba2b0fb8cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:25.612792 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d9ee11-3ff3-4362-9c99-22ba2b0fb8cb admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:25.626766 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1051/2110] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:25.632993 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d9ee11-3ff3-4362-9c99-22ba2b0fb8cb admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:25.637702 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0af6c4-a4b0-4159-bbe2-afc7804e9417 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:25.639235 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0af6c4-a4b0-4159-bbe2-afc7804e9417 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:25.646967 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d9ee11-3ff3-4362-9c99-22ba2b0fb8cb admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:25.657603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1060/2111] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:25.659344 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b072bde-8eff-48c6-bc6d-9e8cc06f6f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:25.660346 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b072bde-8eff-48c6-bc6d-9e8cc06f6f25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:25.661150 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b072bde-8eff-48c6-bc6d-9e8cc06f6f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:25.661150 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b072bde-8eff-48c6-bc6d-9e8cc06f6f25 admin admin] PATH_INFO: `/v3/projects/286b999288bf427bb09c3a2c2ab418a7/users/b6aafb34086b4f8a95a7f52aea356f12/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:25.663417 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b072bde-8eff-48c6-bc6d-9e8cc06f6f25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=286b999288bf427bb09c3a2c2ab418a7, user_id=b6aafb34086b4f8a95a7f52aea356f12, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:25.666211 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b072bde-8eff-48c6-bc6d-9e8cc06f6f25 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:25.677794 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1052/2112] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:25 2022] PUT /identity/v3/projects/286b999288bf427bb09c3a2c2ab418a7/users/b6aafb34086b4f8a95a7f52aea356f12/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:05:25.688114 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94792fc-c6c1-42d8-a49b-2bb752c719f5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:25.688412 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94792fc-c6c1-42d8-a49b-2bb752c719f5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:25.689404 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94792fc-c6c1-42d8-a49b-2bb752c719f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:25.732947 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e94792fc-c6c1-42d8-a49b-2bb752c719f5 None None] MFA Rules not processed for user `b6aafb34086b4f8a95a7f52aea356f12`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:25.735153 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e94792fc-c6c1-42d8-a49b-2bb752c719f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:25.758524 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1f6e4-4659-4cda-946e-ec3c871f1246 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:25.758810 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1f6e4-4659-4cda-946e-ec3c871f1246 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:25.759053 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1f6e4-4659-4cda-946e-ec3c871f1246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:25.804171 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-efa1f6e4-4659-4cda-946e-ec3c871f1246 None None] MFA Rules not processed for user `ad718e6456d542d7a2d99c549909b4c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:25.805409 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-efa1f6e4-4659-4cda-946e-ec3c871f1246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:25.819710 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:25.867099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1061/2113] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:25.867099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94792fc-c6c1-42d8-a49b-2bb752c719f5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:25.867099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e94792fc-c6c1-42d8-a49b-2bb752c719f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:25.893102 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1053/2114] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:05:25.894383 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1f6e4-4659-4cda-946e-ec3c871f1246 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:25.943745 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94792fc-c6c1-42d8-a49b-2bb752c719f5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:25.951919 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d523377-863d-41cf-b957-e90c71844cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:25.952861 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d523377-863d-41cf-b957-e90c71844cde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:25.953077 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d523377-863d-41cf-b957-e90c71844cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:25.953353 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d523377-863d-41cf-b957-e90c71844cde admin admin] PATH_INFO: `/v3/projects/55ae1f94a3bb4f4abe4ab4314030db4f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:25.958704 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d523377-863d-41cf-b957-e90c71844cde admin admin] RBAC: Authorizing `identity:delete_project(project_id=55ae1f94a3bb4f4abe4ab4314030db4f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:25.960809 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d523377-863d-41cf-b957-e90c71844cde admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:25.989913 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1f6e4-4659-4cda-946e-ec3c871f1246 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:25.995563 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf868e-5dd5-4dd1-80a1-bfc4d9fa9bf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:25.996717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf868e-5dd5-4dd1-80a1-bfc4d9fa9bf0 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:25.997190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf868e-5dd5-4dd1-80a1-bfc4d9fa9bf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:25.997528 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf868e-5dd5-4dd1-80a1-bfc4d9fa9bf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:26.009811 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf868e-5dd5-4dd1-80a1-bfc4d9fa9bf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:26.012118 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf868e-5dd5-4dd1-80a1-bfc4d9fa9bf0 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:26.027104 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1054/2115] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:25 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:26.027562 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4d523377-863d-41cf-b957-e90c71844cde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55ae1f94a3bb4f4abe4ab4314030db4f', 'request_id': 'req-4d523377-863d-41cf-b957-e90c71844cde'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:26.030760 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:26.038943 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4d523377-863d-41cf-b957-e90c71844cde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55ae1f94a3bb4f4abe4ab4314030db4f', 'request_id': 'req-4d523377-863d-41cf-b957-e90c71844cde'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:26.048700 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4d523377-863d-41cf-b957-e90c71844cde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55ae1f94a3bb4f4abe4ab4314030db4f', 'request_id': 'req-4d523377-863d-41cf-b957-e90c71844cde'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:26.064697 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4d523377-863d-41cf-b957-e90c71844cde admin admin] The token cache is being invalidate because project 55ae1f94a3bb4f4abe4ab4314030db4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:26.065218 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4d523377-863d-41cf-b957-e90c71844cde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:26.065918 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4d523377-863d-41cf-b957-e90c71844cde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:26.069623 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1062/2116] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:25 2022] DELETE /identity/v3/projects/55ae1f94a3bb4f4abe4ab4314030db4f => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:26.072826 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf51d47-7cba-4a1b-994d-a9e4c0402f14 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:26.073122 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf51d47-7cba-4a1b-994d-a9e4c0402f14 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:26.073404 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf51d47-7cba-4a1b-994d-a9e4c0402f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:26.091018 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d523377-863d-41cf-b957-e90c71844cde admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:26.092723 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4d523377-863d-41cf-b957-e90c71844cde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:26.100006 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-daf51d47-7cba-4a1b-994d-a9e4c0402f14 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:26.101437 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-daf51d47-7cba-4a1b-994d-a9e4c0402f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:26.193182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1055/2117] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:26.199524 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d523377-863d-41cf-b957-e90c71844cde admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:26.201529 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009c793-1ee4-4e23-ab6a-1d5864b228ff None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:26.201848 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009c793-1ee4-4e23-ab6a-1d5864b228ff None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:26.202131 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009c793-1ee4-4e23-ab6a-1d5864b228ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:26.204399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb950c-fd83-4beb-a83f-b71649b80c24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:26.205850 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb950c-fd83-4beb-a83f-b71649b80c24 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:26.209656 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb950c-fd83-4beb-a83f-b71649b80c24 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:26.209656 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb950c-fd83-4beb-a83f-b71649b80c24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:26.209656 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6bcb950c-fd83-4beb-a83f-b71649b80c24 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:26.226041 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3009c793-1ee4-4e23-ab6a-1d5864b228ff None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:26.228091 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3009c793-1ee4-4e23-ab6a-1d5864b228ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:26.247364 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1056/2118] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:26.251478 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009c793-1ee4-4e23-ab6a-1d5864b228ff None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:26.260829 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009c793-1ee4-4e23-ab6a-1d5864b228ff None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:26.266629 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e0ce5-692e-4525-97d1-e6623d56167a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:26.267673 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e0ce5-692e-4525-97d1-e6623d56167a admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:26.267961 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e0ce5-692e-4525-97d1-e6623d56167a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:26.268271 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e0ce5-692e-4525-97d1-e6623d56167a admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:26.268982 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049e0ce5-692e-4525-97d1-e6623d56167a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:05:26.269294 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049e0ce5-692e-4525-97d1-e6623d56167a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:26.272187 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049e0ce5-692e-4525-97d1-e6623d56167a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:26.282262 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcb950c-fd83-4beb-a83f-b71649b80c24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:26.284178 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcb950c-fd83-4beb-a83f-b71649b80c24 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:26.286490 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1057/2119] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:05:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 16:05:26.291462 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e0ce5-692e-4525-97d1-e6623d56167a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:26.300284 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1063/2120] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:26.302556 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e0ce5-692e-4525-97d1-e6623d56167a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:26.308727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb30a85-e869-4cff-bbc8-0163ccb99ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:26.309408 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb30a85-e869-4cff-bbc8-0163ccb99ebd admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:26.309726 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb30a85-e869-4cff-bbc8-0163ccb99ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:26.310026 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb30a85-e869-4cff-bbc8-0163ccb99ebd admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:26.310742 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb30a85-e869-4cff-bbc8-0163ccb99ebd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:26.312652 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb30a85-e869-4cff-bbc8-0163ccb99ebd admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:26.357092 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1058/2121] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:26 2022] POST /identity/v3/projects => generated 371 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:05:26.361851 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb950c-fd83-4beb-a83f-b71649b80c24 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:26.377001 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb950c-fd83-4beb-a83f-b71649b80c24 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:26.448285 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f0721-2272-4932-b6a0-9ddf38718be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:26.448974 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f0721-2272-4932-b6a0-9ddf38718be9 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:26.449259 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f0721-2272-4932-b6a0-9ddf38718be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:26.449604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f0721-2272-4932-b6a0-9ddf38718be9 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:26.450316 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f0721-2272-4932-b6a0-9ddf38718be9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:26.452257 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f0721-2272-4932-b6a0-9ddf38718be9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:26.473938 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1064/2122] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:26 2022] POST /identity/v3/users => generated 286 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:05:26.478940 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb30a85-e869-4cff-bbc8-0163ccb99ebd admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:26.492235 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb30a85-e869-4cff-bbc8-0163ccb99ebd admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:26.498406 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e66a8d-3e1a-41cc-a2b7-0b452b05e3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:26.499089 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e66a8d-3e1a-41cc-a2b7-0b452b05e3d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:26.499376 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e66a8d-3e1a-41cc-a2b7-0b452b05e3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:26.499693 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e66a8d-3e1a-41cc-a2b7-0b452b05e3d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:26.500414 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e66a8d-3e1a-41cc-a2b7-0b452b05e3d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:26.502258 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e66a8d-3e1a-41cc-a2b7-0b452b05e3d0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:26.517855 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1059/2123] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:26.519780 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f0721-2272-4932-b6a0-9ddf38718be9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:26.533977 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f0721-2272-4932-b6a0-9ddf38718be9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:26.539831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda6e0e0-bf94-439f-a060-c10adb4a45d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:26.540713 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6e0e0-bf94-439f-a060-c10adb4a45d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:26.540843 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6e0e0-bf94-439f-a060-c10adb4a45d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:26.541055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6e0e0-bf94-439f-a060-c10adb4a45d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:26.541886 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda6e0e0-bf94-439f-a060-c10adb4a45d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:26.543455 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda6e0e0-bf94-439f-a060-c10adb4a45d3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:26.554836 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1065/2124] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:26.559496 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e66a8d-3e1a-41cc-a2b7-0b452b05e3d0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:26.566047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:26.573005 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e66a8d-3e1a-41cc-a2b7-0b452b05e3d0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:26.588750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c08e95-1c9e-48ab-8d4e-d58b72c91832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:26.589857 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c08e95-1c9e-48ab-8d4e-d58b72c91832 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:26.590146 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c08e95-1c9e-48ab-8d4e-d58b72c91832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:26.590424 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c08e95-1c9e-48ab-8d4e-d58b72c91832 admin admin] PATH_INFO: `/v3/projects/505b62f973f640c5813e52b1c0454e40/users/9cacf7e1548b46a2bae7cc0d431b24f6/roles/54195e660fc349978b39d5794765a095` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:26.628439 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c08e95-1c9e-48ab-8d4e-d58b72c91832 admin admin] RBAC: Authorizing `identity:create_grant(project_id=505b62f973f640c5813e52b1c0454e40, user_id=9cacf7e1548b46a2bae7cc0d431b24f6, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:26.631944 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c08e95-1c9e-48ab-8d4e-d58b72c91832 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:26.649188 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1060/2125] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:26 2022] PUT /identity/v3/projects/505b62f973f640c5813e52b1c0454e40/users/9cacf7e1548b46a2bae7cc0d431b24f6/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:26.664466 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7d031-9d01-4946-8c7b-6a30c8c70060 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:26.664466 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7d031-9d01-4946-8c7b-6a30c8c70060 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:26.664466 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7d031-9d01-4946-8c7b-6a30c8c70060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:26.694216 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:26.700388 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2ea7d031-9d01-4946-8c7b-6a30c8c70060 None None] MFA Rules not processed for user `9cacf7e1548b46a2bae7cc0d431b24f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:26.701646 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2ea7d031-9d01-4946-8c7b-6a30c8c70060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:26.771676 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c08e95-1c9e-48ab-8d4e-d58b72c91832 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:26.775222 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c4c08e95-1c9e-48ab-8d4e-d58b72c91832 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:26.800218 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1066/2126] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:26.838423 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c08e95-1c9e-48ab-8d4e-d58b72c91832 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:26.842652 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fef047b-ff61-4b93-84fe-cd56a563a172 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:26.843142 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef047b-ff61-4b93-84fe-cd56a563a172 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:26.843428 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef047b-ff61-4b93-84fe-cd56a563a172 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:26.843734 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef047b-ff61-4b93-84fe-cd56a563a172 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:26.845496 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1fef047b-ff61-4b93-84fe-cd56a563a172 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:26.877878 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7d031-9d01-4946-8c7b-6a30c8c70060 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:26.888942 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7d031-9d01-4946-8c7b-6a30c8c70060 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:26.892880 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efef54-747a-4aee-b434-bf00f7ebc132 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:26.893523 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efef54-747a-4aee-b434-bf00f7ebc132 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:26.893841 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efef54-747a-4aee-b434-bf00f7ebc132 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:26.894115 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efef54-747a-4aee-b434-bf00f7ebc132 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:26.904084 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68efef54-747a-4aee-b434-bf00f7ebc132 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:26.905525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68efef54-747a-4aee-b434-bf00f7ebc132 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:26.920358 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fef047b-ff61-4b93-84fe-cd56a563a172 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:26.921787 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1067/2127] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 16:05:26.922077 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fef047b-ff61-4b93-84fe-cd56a563a172 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:26.940928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1061/2128] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 16:05:26.942982 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efef54-747a-4aee-b434-bf00f7ebc132 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:27.026183 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efef54-747a-4aee-b434-bf00f7ebc132 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:27.031543 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfe2556-c2be-412d-b87a-a0741a3123bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:27.032452 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe2556-c2be-412d-b87a-a0741a3123bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:27.032738 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe2556-c2be-412d-b87a-a0741a3123bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:27.033017 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe2556-c2be-412d-b87a-a0741a3123bc admin admin] PATH_INFO: `/v3/users/9cacf7e1548b46a2bae7cc0d431b24f6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:27.034251 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfe2556-c2be-412d-b87a-a0741a3123bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cacf7e1548b46a2bae7cc0d431b24f6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:27.036189 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfe2556-c2be-412d-b87a-a0741a3123bc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:27.086680 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4dfe2556-c2be-412d-b87a-a0741a3123bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cacf7e1548b46a2bae7cc0d431b24f6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:27.087601 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4dfe2556-c2be-412d-b87a-a0741a3123bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cacf7e1548b46a2bae7cc0d431b24f6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:27.088350 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4dfe2556-c2be-412d-b87a-a0741a3123bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cacf7e1548b46a2bae7cc0d431b24f6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:27.096950 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4dfe2556-c2be-412d-b87a-a0741a3123bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cacf7e1548b46a2bae7cc0d431b24f6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:27.105161 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4dfe2556-c2be-412d-b87a-a0741a3123bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cacf7e1548b46a2bae7cc0d431b24f6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:27.124099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1068/2129] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:26 2022] DELETE /identity/v3/users/9cacf7e1548b46a2bae7cc0d431b24f6 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:27.130143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e800642-a511-41b0-81dc-e62c9ab9b712 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:27.131089 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e800642-a511-41b0-81dc-e62c9ab9b712 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:27.131453 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e800642-a511-41b0-81dc-e62c9ab9b712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:27.150333 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2e800642-a511-41b0-81dc-e62c9ab9b712 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:27.151581 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2e800642-a511-41b0-81dc-e62c9ab9b712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:27.227875 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1062/2130] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:27.246708 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfe2556-c2be-412d-b87a-a0741a3123bc admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:27.250787 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4dfe2556-c2be-412d-b87a-a0741a3123bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:27.277153 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:27.346466 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfe2556-c2be-412d-b87a-a0741a3123bc admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:27.350526 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129240cc-dfc1-4943-84a4-899f7196b73b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:27.351241 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129240cc-dfc1-4943-84a4-899f7196b73b service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:27.351593 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129240cc-dfc1-4943-84a4-899f7196b73b service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:27.351955 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129240cc-dfc1-4943-84a4-899f7196b73b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:27.362323 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129240cc-dfc1-4943-84a4-899f7196b73b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:27.364314 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129240cc-dfc1-4943-84a4-899f7196b73b service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:27.375196 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1069/2131] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:27.674684 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e800642-a511-41b0-81dc-e62c9ab9b712 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:27.676204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2e800642-a511-41b0-81dc-e62c9ab9b712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:27.707276 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e800642-a511-41b0-81dc-e62c9ab9b712 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:27.713376 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993d8e82-04f5-4e36-a635-5540562511b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:27.714331 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d8e82-04f5-4e36-a635-5540562511b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:27.714621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d8e82-04f5-4e36-a635-5540562511b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:27.714902 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d8e82-04f5-4e36-a635-5540562511b6 admin admin] PATH_INFO: `/v3/projects/505b62f973f640c5813e52b1c0454e40` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:27.716221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993d8e82-04f5-4e36-a635-5540562511b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=505b62f973f640c5813e52b1c0454e40)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:27.718256 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993d8e82-04f5-4e36-a635-5540562511b6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:27.747765 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5e1b8-7479-4726-a81d-33210ee8619d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:27.748082 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5e1b8-7479-4726-a81d-33210ee8619d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:27.748434 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5e1b8-7479-4726-a81d-33210ee8619d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:27.756447 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:27.767211 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-aed5e1b8-7479-4726-a81d-33210ee8619d None None] MFA Rules not processed for user `ad718e6456d542d7a2d99c549909b4c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:27.768471 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-aed5e1b8-7479-4726-a81d-33210ee8619d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:27.772976 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-993d8e82-04f5-4e36-a635-5540562511b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '505b62f973f640c5813e52b1c0454e40', 'request_id': 'req-993d8e82-04f5-4e36-a635-5540562511b6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:27.780616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-993d8e82-04f5-4e36-a635-5540562511b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '505b62f973f640c5813e52b1c0454e40', 'request_id': 'req-993d8e82-04f5-4e36-a635-5540562511b6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:27.787604 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-993d8e82-04f5-4e36-a635-5540562511b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '505b62f973f640c5813e52b1c0454e40', 'request_id': 'req-993d8e82-04f5-4e36-a635-5540562511b6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:27.798874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-993d8e82-04f5-4e36-a635-5540562511b6 admin admin] The token cache is being invalidate because project 505b62f973f640c5813e52b1c0454e40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:27.799169 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-993d8e82-04f5-4e36-a635-5540562511b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:27.799815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-993d8e82-04f5-4e36-a635-5540562511b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:27.801414 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1063/2132] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:27 2022] DELETE /identity/v3/projects/505b62f973f640c5813e52b1c0454e40 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:27.827899 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1070/2133] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:05:30.139348 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993d8e82-04f5-4e36-a635-5540562511b6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:30.140771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-993d8e82-04f5-4e36-a635-5540562511b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:30.198913 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993d8e82-04f5-4e36-a635-5540562511b6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:30.202485 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90376b2-2e9f-4a54-85f6-0e72511d1f3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:30.203120 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90376b2-2e9f-4a54-85f6-0e72511d1f3a service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:30.203451 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90376b2-2e9f-4a54-85f6-0e72511d1f3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:30.203732 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90376b2-2e9f-4a54-85f6-0e72511d1f3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:30.205491 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b90376b2-2e9f-4a54-85f6-0e72511d1f3a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:30.270313 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90376b2-2e9f-4a54-85f6-0e72511d1f3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:30.271833 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90376b2-2e9f-4a54-85f6-0e72511d1f3a service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:30.282267 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1064/2134] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:30 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:31.148377 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5e1b8-7479-4726-a81d-33210ee8619d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:31.149874 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-aed5e1b8-7479-4726-a81d-33210ee8619d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:31.161907 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5e1b8-7479-4726-a81d-33210ee8619d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:31.166984 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff345650-85c2-4c0e-a990-3642a2dcd700 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:31.167684 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff345650-85c2-4c0e-a990-3642a2dcd700 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:31.167971 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff345650-85c2-4c0e-a990-3642a2dcd700 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:31.168249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff345650-85c2-4c0e-a990-3642a2dcd700 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:31.170060 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ff345650-85c2-4c0e-a990-3642a2dcd700 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:31.244551 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff345650-85c2-4c0e-a990-3642a2dcd700 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:31.249720 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff345650-85c2-4c0e-a990-3642a2dcd700 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:31.262232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1071/2135] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:33.851744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90376b2-2e9f-4a54-85f6-0e72511d1f3a service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:33.853184 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b90376b2-2e9f-4a54-85f6-0e72511d1f3a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:33.923871 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90376b2-2e9f-4a54-85f6-0e72511d1f3a service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:33.928520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f0ca89-489c-4394-bf07-cf17951dc65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:33.929406 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f0ca89-489c-4394-bf07-cf17951dc65b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:33.929724 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f0ca89-489c-4394-bf07-cf17951dc65b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:33.930012 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f0ca89-489c-4394-bf07-cf17951dc65b admin admin] PATH_INFO: `/v3/users/b8ec2f6793df41428a48cf13fe1c4cc4` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:33.931207 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f0ca89-489c-4394-bf07-cf17951dc65b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8ec2f6793df41428a48cf13fe1c4cc4)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:33.933232 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f0ca89-489c-4394-bf07-cf17951dc65b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:33.945163 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:33.992901 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-54f0ca89-489c-4394-bf07-cf17951dc65b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8ec2f6793df41428a48cf13fe1c4cc4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:33.993595 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-54f0ca89-489c-4394-bf07-cf17951dc65b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8ec2f6793df41428a48cf13fe1c4cc4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:33.994254 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-54f0ca89-489c-4394-bf07-cf17951dc65b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8ec2f6793df41428a48cf13fe1c4cc4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:34.004253 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-54f0ca89-489c-4394-bf07-cf17951dc65b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8ec2f6793df41428a48cf13fe1c4cc4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:34.020856 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-54f0ca89-489c-4394-bf07-cf17951dc65b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8ec2f6793df41428a48cf13fe1c4cc4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:34.067962 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1065/2136] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:33 2022] DELETE /identity/v3/users/b8ec2f6793df41428a48cf13fe1c4cc4 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:34.070359 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daed510-4343-4a11-a872-29dc836807a5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:34.070359 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daed510-4343-4a11-a872-29dc836807a5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:34.071026 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daed510-4343-4a11-a872-29dc836807a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:34.106150 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3daed510-4343-4a11-a872-29dc836807a5 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:34.107689 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3daed510-4343-4a11-a872-29dc836807a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:34.201275 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:34.216333 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1072/2137] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:05:34.227480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f0ca89-489c-4394-bf07-cf17951dc65b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:34.229323 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-54f0ca89-489c-4394-bf07-cf17951dc65b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:34.312362 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f0ca89-489c-4394-bf07-cf17951dc65b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:34.316483 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697cf7ca-92d9-4684-bd62-be4e3f7e859b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:34.317267 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697cf7ca-92d9-4684-bd62-be4e3f7e859b service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:34.317548 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697cf7ca-92d9-4684-bd62-be4e3f7e859b service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:34.317856 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697cf7ca-92d9-4684-bd62-be4e3f7e859b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:34.327895 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697cf7ca-92d9-4684-bd62-be4e3f7e859b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:34.329320 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697cf7ca-92d9-4684-bd62-be4e3f7e859b service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:34.339670 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1066/2138] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:34.614683 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daed510-4343-4a11-a872-29dc836807a5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:34.616292 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3daed510-4343-4a11-a872-29dc836807a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:34.628621 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daed510-4343-4a11-a872-29dc836807a5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:34.634406 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a501673a-d042-4e17-a0fb-ff8ca8f88d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:34.635271 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a501673a-d042-4e17-a0fb-ff8ca8f88d1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:34.635557 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a501673a-d042-4e17-a0fb-ff8ca8f88d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:34.635836 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a501673a-d042-4e17-a0fb-ff8ca8f88d1d admin admin] PATH_INFO: `/v3/projects/18f6ce66d6834a13b6cacd807445af1a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:34.637023 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a501673a-d042-4e17-a0fb-ff8ca8f88d1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=18f6ce66d6834a13b6cacd807445af1a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:34.638987 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a501673a-d042-4e17-a0fb-ff8ca8f88d1d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:34.685159 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a501673a-d042-4e17-a0fb-ff8ca8f88d1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18f6ce66d6834a13b6cacd807445af1a', 'request_id': 'req-a501673a-d042-4e17-a0fb-ff8ca8f88d1d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:34.692290 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a501673a-d042-4e17-a0fb-ff8ca8f88d1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18f6ce66d6834a13b6cacd807445af1a', 'request_id': 'req-a501673a-d042-4e17-a0fb-ff8ca8f88d1d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:34.698849 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a501673a-d042-4e17-a0fb-ff8ca8f88d1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18f6ce66d6834a13b6cacd807445af1a', 'request_id': 'req-a501673a-d042-4e17-a0fb-ff8ca8f88d1d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:34.710254 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a501673a-d042-4e17-a0fb-ff8ca8f88d1d admin admin] The token cache is being invalidate because project 18f6ce66d6834a13b6cacd807445af1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:34.710540 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a501673a-d042-4e17-a0fb-ff8ca8f88d1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:34.711468 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a501673a-d042-4e17-a0fb-ff8ca8f88d1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:34.713030 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1073/2139] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:34 2022] DELETE /identity/v3/projects/18f6ce66d6834a13b6cacd807445af1a => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:34.717252 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697cf7ca-92d9-4684-bd62-be4e3f7e859b service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:34.727407 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697cf7ca-92d9-4684-bd62-be4e3f7e859b service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:34.794323 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b2489-8c9b-45c9-8e06-69ef55710f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:34.794323 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b2489-8c9b-45c9-8e06-69ef55710f9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:34.794323 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b2489-8c9b-45c9-8e06-69ef55710f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:34.794323 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b2489-8c9b-45c9-8e06-69ef55710f9b admin admin] PATH_INFO: `/v3/users/daf943301b734d17ab6c4521271a3de5` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:34.794903 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834b2489-8c9b-45c9-8e06-69ef55710f9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=daf943301b734d17ab6c4521271a3de5)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:34.796743 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834b2489-8c9b-45c9-8e06-69ef55710f9b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:34.837805 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:34.854424 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-834b2489-8c9b-45c9-8e06-69ef55710f9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'daf943301b734d17ab6c4521271a3de5'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:34.855098 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-834b2489-8c9b-45c9-8e06-69ef55710f9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'daf943301b734d17ab6c4521271a3de5'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:34.856189 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-834b2489-8c9b-45c9-8e06-69ef55710f9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'daf943301b734d17ab6c4521271a3de5'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:34.867655 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-834b2489-8c9b-45c9-8e06-69ef55710f9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'daf943301b734d17ab6c4521271a3de5'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:34.875799 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-834b2489-8c9b-45c9-8e06-69ef55710f9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'daf943301b734d17ab6c4521271a3de5'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:34.899710 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1067/2140] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:34 2022] DELETE /identity/v3/users/daf943301b734d17ab6c4521271a3de5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:35.185483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a501673a-d042-4e17-a0fb-ff8ca8f88d1d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:35.186956 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a501673a-d042-4e17-a0fb-ff8ca8f88d1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:35.207363 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a501673a-d042-4e17-a0fb-ff8ca8f88d1d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:35.227587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b2489-8c9b-45c9-8e06-69ef55710f9b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:35.229391 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-834b2489-8c9b-45c9-8e06-69ef55710f9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:35.282367 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e97c53-0cbf-4b33-bde6-d068bbf67469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:35.283276 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e97c53-0cbf-4b33-bde6-d068bbf67469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:35.283576 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e97c53-0cbf-4b33-bde6-d068bbf67469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:35.283908 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e97c53-0cbf-4b33-bde6-d068bbf67469 admin admin] PATH_INFO: `/v3/projects/8ecda84209174d5c884f7a6417449d63` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:35.285399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e97c53-0cbf-4b33-bde6-d068bbf67469 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ecda84209174d5c884f7a6417449d63)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:35.288035 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e97c53-0cbf-4b33-bde6-d068bbf67469 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:35.304662 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b2489-8c9b-45c9-8e06-69ef55710f9b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:35.309936 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a68eea-6a15-4445-b848-46a4c3192de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:35.310803 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a68eea-6a15-4445-b848-46a4c3192de0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:35.311087 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a68eea-6a15-4445-b848-46a4c3192de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:35.311823 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a68eea-6a15-4445-b848-46a4c3192de0 admin admin] PATH_INFO: `/v3/users/afc50f7c8b444a61a3a43b2aa8ea540e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:35.313068 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a68eea-6a15-4445-b848-46a4c3192de0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afc50f7c8b444a61a3a43b2aa8ea540e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:35.315243 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a68eea-6a15-4445-b848-46a4c3192de0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:35.341965 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d4e97c53-0cbf-4b33-bde6-d068bbf67469 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ecda84209174d5c884f7a6417449d63', 'request_id': 'req-d4e97c53-0cbf-4b33-bde6-d068bbf67469'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:35.353662 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d4e97c53-0cbf-4b33-bde6-d068bbf67469 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ecda84209174d5c884f7a6417449d63', 'request_id': 'req-d4e97c53-0cbf-4b33-bde6-d068bbf67469'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:35.362046 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d4e97c53-0cbf-4b33-bde6-d068bbf67469 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ecda84209174d5c884f7a6417449d63', 'request_id': 'req-d4e97c53-0cbf-4b33-bde6-d068bbf67469'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:35.373368 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-58a68eea-6a15-4445-b848-46a4c3192de0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afc50f7c8b444a61a3a43b2aa8ea540e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:35.374358 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-58a68eea-6a15-4445-b848-46a4c3192de0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afc50f7c8b444a61a3a43b2aa8ea540e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:35.375148 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-58a68eea-6a15-4445-b848-46a4c3192de0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afc50f7c8b444a61a3a43b2aa8ea540e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:35.375632 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d4e97c53-0cbf-4b33-bde6-d068bbf67469 admin admin] The token cache is being invalidate because project 8ecda84209174d5c884f7a6417449d63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:35.376110 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d4e97c53-0cbf-4b33-bde6-d068bbf67469 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:35.377019 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d4e97c53-0cbf-4b33-bde6-d068bbf67469 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:35.379130 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1074/2141] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:35 2022] DELETE /identity/v3/projects/8ecda84209174d5c884f7a6417449d63 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:35.389750 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e97c53-0cbf-4b33-bde6-d068bbf67469 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:35.393179 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d4e97c53-0cbf-4b33-bde6-d068bbf67469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:35.395829 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-58a68eea-6a15-4445-b848-46a4c3192de0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afc50f7c8b444a61a3a43b2aa8ea540e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:35.406824 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-58a68eea-6a15-4445-b848-46a4c3192de0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afc50f7c8b444a61a3a43b2aa8ea540e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:35.417272 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:35.430910 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1068/2142] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:35 2022] DELETE /identity/v3/users/afc50f7c8b444a61a3a43b2aa8ea540e => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:05:35.443982 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b1ac9-e24d-49ec-8ffc-b268e3d59330 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:35.444487 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b1ac9-e24d-49ec-8ffc-b268e3d59330 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:35.445252 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b1ac9-e24d-49ec-8ffc-b268e3d59330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:35.469511 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5b8b1ac9-e24d-49ec-8ffc-b268e3d59330 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:35.471192 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5b8b1ac9-e24d-49ec-8ffc-b268e3d59330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:35.500993 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e97c53-0cbf-4b33-bde6-d068bbf67469 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:35.509411 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:35.563865 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1069/2143] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:35.566597 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa320d1-0583-47fb-bd65-e848da4df324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:35.567502 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa320d1-0583-47fb-bd65-e848da4df324 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:35.567767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa320d1-0583-47fb-bd65-e848da4df324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:35.568074 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa320d1-0583-47fb-bd65-e848da4df324 admin admin] PATH_INFO: `/v3/users/b6aafb34086b4f8a95a7f52aea356f12` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:35.569448 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa320d1-0583-47fb-bd65-e848da4df324 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6aafb34086b4f8a95a7f52aea356f12)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:35.571506 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa320d1-0583-47fb-bd65-e848da4df324 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:35.575979 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b1ac9-e24d-49ec-8ffc-b268e3d59330 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:35.578206 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5b8b1ac9-e24d-49ec-8ffc-b268e3d59330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:35.640439 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eaa320d1-0583-47fb-bd65-e848da4df324 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6aafb34086b4f8a95a7f52aea356f12'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:35.641124 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eaa320d1-0583-47fb-bd65-e848da4df324 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6aafb34086b4f8a95a7f52aea356f12'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:35.641877 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eaa320d1-0583-47fb-bd65-e848da4df324 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6aafb34086b4f8a95a7f52aea356f12'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:35.649559 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b1ac9-e24d-49ec-8ffc-b268e3d59330 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:35.652754 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eaa320d1-0583-47fb-bd65-e848da4df324 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6aafb34086b4f8a95a7f52aea356f12'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:35.653748 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94051a7-c5b8-4e4e-acf0-83434b80e685 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:35.654469 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94051a7-c5b8-4e4e-acf0-83434b80e685 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:35.654819 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94051a7-c5b8-4e4e-acf0-83434b80e685 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:35.655167 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94051a7-c5b8-4e4e-acf0-83434b80e685 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:35.657089 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a94051a7-c5b8-4e4e-acf0-83434b80e685 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:35.662071 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eaa320d1-0583-47fb-bd65-e848da4df324 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6aafb34086b4f8a95a7f52aea356f12'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:35.672177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94051a7-c5b8-4e4e-acf0-83434b80e685 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:35.673784 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94051a7-c5b8-4e4e-acf0-83434b80e685 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:35.686700 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1070/2144] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:35.692442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1075/2145] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:35 2022] DELETE /identity/v3/users/b6aafb34086b4f8a95a7f52aea356f12 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:36.572798 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3e1d2-605a-4318-a855-9f3c160ba057 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:36.573712 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3e1d2-605a-4318-a855-9f3c160ba057 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:36.574312 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3e1d2-605a-4318-a855-9f3c160ba057 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:36.576359 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1071/2146] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jul 21 16:05:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 16:05:36.615953 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8e307-54bc-4552-9f51-20286baf2ffd None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:36.616543 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8e307-54bc-4552-9f51-20286baf2ffd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:36.616881 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8e307-54bc-4552-9f51-20286baf2ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:36.645709 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b3d8e307-54bc-4552-9f51-20286baf2ffd None None] MFA Rules not processed for user `822d9d3521db4a9688f51c6313c7dd74`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:36.647573 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b3d8e307-54bc-4552-9f51-20286baf2ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:36.672515 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3e1d2-605a-4318-a855-9f3c160ba057 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:36.675242 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0aa3e1d2-605a-4318-a855-9f3c160ba057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:36.717518 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:36.775214 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1076/2147] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jul 21 16:05:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:36.776395 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8e307-54bc-4552-9f51-20286baf2ffd None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:36.777945 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b3d8e307-54bc-4552-9f51-20286baf2ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:36.820931 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3e1d2-605a-4318-a855-9f3c160ba057 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:36.829698 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d695e-3f9c-4bd6-b232-958ffaa14a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:36.830870 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d695e-3f9c-4bd6-b232-958ffaa14a0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:36.831189 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d695e-3f9c-4bd6-b232-958ffaa14a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:36.831870 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d695e-3f9c-4bd6-b232-958ffaa14a0d admin admin] PATH_INFO: `/v3/projects/286b999288bf427bb09c3a2c2ab418a7` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:36.834788 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621d695e-3f9c-4bd6-b232-958ffaa14a0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=286b999288bf427bb09c3a2c2ab418a7)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:36.838382 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621d695e-3f9c-4bd6-b232-958ffaa14a0d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:36.879669 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8e307-54bc-4552-9f51-20286baf2ffd None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:36.886009 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d47343-0d80-4071-a35d-5b55e670c36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:36.887272 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d47343-0d80-4071-a35d-5b55e670c36c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:36.887768 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d47343-0d80-4071-a35d-5b55e670c36c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:36.888314 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d47343-0d80-4071-a35d-5b55e670c36c admin admin] PATH_INFO: `/v3/projects/5050d95145f94d63bbe3a5970907bbe7` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:36.889868 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d47343-0d80-4071-a35d-5b55e670c36c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5050d95145f94d63bbe3a5970907bbe7)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:36.892126 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d47343-0d80-4071-a35d-5b55e670c36c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:36.906227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-621d695e-3f9c-4bd6-b232-958ffaa14a0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '286b999288bf427bb09c3a2c2ab418a7', 'request_id': 'req-621d695e-3f9c-4bd6-b232-958ffaa14a0d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:36.918790 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-621d695e-3f9c-4bd6-b232-958ffaa14a0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '286b999288bf427bb09c3a2c2ab418a7', 'request_id': 'req-621d695e-3f9c-4bd6-b232-958ffaa14a0d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:36.933624 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-621d695e-3f9c-4bd6-b232-958ffaa14a0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '286b999288bf427bb09c3a2c2ab418a7', 'request_id': 'req-621d695e-3f9c-4bd6-b232-958ffaa14a0d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:36.952366 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-621d695e-3f9c-4bd6-b232-958ffaa14a0d admin admin] The token cache is being invalidate because project 286b999288bf427bb09c3a2c2ab418a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:36.952661 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-621d695e-3f9c-4bd6-b232-958ffaa14a0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:36.956160 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-621d695e-3f9c-4bd6-b232-958ffaa14a0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:36.958168 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1072/2148] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:36 2022] DELETE /identity/v3/projects/286b999288bf427bb09c3a2c2ab418a7 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:36.959033 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d695e-3f9c-4bd6-b232-958ffaa14a0d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:36.960437 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-621d695e-3f9c-4bd6-b232-958ffaa14a0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:36.966543 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:36.993507 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-94d47343-0d80-4071-a35d-5b55e670c36c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5050d95145f94d63bbe3a5970907bbe7', 'request_id': 'req-94d47343-0d80-4071-a35d-5b55e670c36c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:37.004047 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-94d47343-0d80-4071-a35d-5b55e670c36c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5050d95145f94d63bbe3a5970907bbe7', 'request_id': 'req-94d47343-0d80-4071-a35d-5b55e670c36c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:37.015036 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-94d47343-0d80-4071-a35d-5b55e670c36c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5050d95145f94d63bbe3a5970907bbe7', 'request_id': 'req-94d47343-0d80-4071-a35d-5b55e670c36c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:37.034836 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-94d47343-0d80-4071-a35d-5b55e670c36c admin admin] The token cache is being invalidate because project 5050d95145f94d63bbe3a5970907bbe7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:37.035151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-94d47343-0d80-4071-a35d-5b55e670c36c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:37.035971 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-94d47343-0d80-4071-a35d-5b55e670c36c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:37.038872 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1077/2149] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:36 2022] DELETE /identity/v3/projects/5050d95145f94d63bbe3a5970907bbe7 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:37.040341 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4a514-b65a-4ad0-a671-148c4f0286e3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:37.040706 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4a514-b65a-4ad0-a671-148c4f0286e3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:37.041048 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4a514-b65a-4ad0-a671-148c4f0286e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:37.061076 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d695e-3f9c-4bd6-b232-958ffaa14a0d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:37.061712 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-12e4a514-b65a-4ad0-a671-148c4f0286e3 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:37.063260 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-12e4a514-b65a-4ad0-a671-148c4f0286e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:37.133210 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1078/2150] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:37.134162 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e4a514-b65a-4ad0-a671-148c4f0286e3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:37.135451 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-12e4a514-b65a-4ad0-a671-148c4f0286e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:37.142525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:37.149295 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e4a514-b65a-4ad0-a671-148c4f0286e3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:37.157835 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383e6e8-e345-417c-bc6d-e0a49c58ac40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:37.158964 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383e6e8-e345-417c-bc6d-e0a49c58ac40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:37.159495 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383e6e8-e345-417c-bc6d-e0a49c58ac40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:37.159932 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383e6e8-e345-417c-bc6d-e0a49c58ac40 admin admin] PATH_INFO: `/v3/users/9316a2fa67fc4452ad2809ed43bbf585` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:37.161273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0383e6e8-e345-417c-bc6d-e0a49c58ac40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9316a2fa67fc4452ad2809ed43bbf585)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:37.164037 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0383e6e8-e345-417c-bc6d-e0a49c58ac40 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:37.166021 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838642c-bf06-4190-91cb-59e90e9e3363 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822d9d3521db4a9688f51c6313c7dd74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:37.167209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838642c-bf06-4190-91cb-59e90e9e3363 service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:37.168072 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838642c-bf06-4190-91cb-59e90e9e3363 service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:37.168714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838642c-bf06-4190-91cb-59e90e9e3363 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:37.172132 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f838642c-bf06-4190-91cb-59e90e9e3363 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:37.187137 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f838642c-bf06-4190-91cb-59e90e9e3363 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:37.188959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f838642c-bf06-4190-91cb-59e90e9e3363 service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:37.202619 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1073/2151] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:37.203911 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8331a-7773-413c-aa93-962bae03788e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:37.204212 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8331a-7773-413c-aa93-962bae03788e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:37.204499 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8331a-7773-413c-aa93-962bae03788e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:37.223839 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0383e6e8-e345-417c-bc6d-e0a49c58ac40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9316a2fa67fc4452ad2809ed43bbf585'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:37.224920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0383e6e8-e345-417c-bc6d-e0a49c58ac40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9316a2fa67fc4452ad2809ed43bbf585'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:37.225218 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0383e6e8-e345-417c-bc6d-e0a49c58ac40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9316a2fa67fc4452ad2809ed43bbf585'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:37.238967 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a1b8331a-7773-413c-aa93-962bae03788e None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:37.240087 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a1b8331a-7773-413c-aa93-962bae03788e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:37.243727 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0383e6e8-e345-417c-bc6d-e0a49c58ac40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9316a2fa67fc4452ad2809ed43bbf585'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:37.255910 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1074/2152] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:37.258527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0383e6e8-e345-417c-bc6d-e0a49c58ac40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9316a2fa67fc4452ad2809ed43bbf585'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:37.271340 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8331a-7773-413c-aa93-962bae03788e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:37.283400 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1079/2153] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:37 2022] DELETE /identity/v3/users/9316a2fa67fc4452ad2809ed43bbf585 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:37.283805 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8331a-7773-413c-aa93-962bae03788e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:37.352144 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383e6e8-e345-417c-bc6d-e0a49c58ac40 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:37.353518 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0383e6e8-e345-417c-bc6d-e0a49c58ac40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:37.359944 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1508cf7c-c050-4f59-88d6-f093c008254e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:37.360523 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508cf7c-c050-4f59-88d6-f093c008254e admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:37.360813 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508cf7c-c050-4f59-88d6-f093c008254e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:37.361219 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508cf7c-c050-4f59-88d6-f093c008254e admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:37.362076 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1508cf7c-c050-4f59-88d6-f093c008254e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:05:37.362465 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1508cf7c-c050-4f59-88d6-f093c008254e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:37.365169 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1508cf7c-c050-4f59-88d6-f093c008254e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:37.378034 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383e6e8-e345-417c-bc6d-e0a49c58ac40 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:37.385623 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1075/2154] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:05:37.385822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730e975-2c73-4d4b-b971-58de6c8e373f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:37.386191 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730e975-2c73-4d4b-b971-58de6c8e373f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:37.386548 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730e975-2c73-4d4b-b971-58de6c8e373f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:37.386935 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730e975-2c73-4d4b-b971-58de6c8e373f admin admin] PATH_INFO: `/v3/users/8c28991387694c958cdd7a98e4ce12ae` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:37.389643 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730e975-2c73-4d4b-b971-58de6c8e373f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c28991387694c958cdd7a98e4ce12ae)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:37.389978 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1508cf7c-c050-4f59-88d6-f093c008254e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:37.393704 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730e975-2c73-4d4b-b971-58de6c8e373f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:37.404959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1508cf7c-c050-4f59-88d6-f093c008254e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:37.412817 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e934f3f6-8490-4b90-951e-c9db5fdc00b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:37.413992 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934f3f6-8490-4b90-951e-c9db5fdc00b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:37.414278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934f3f6-8490-4b90-951e-c9db5fdc00b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:37.414671 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934f3f6-8490-4b90-951e-c9db5fdc00b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:37.415349 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e934f3f6-8490-4b90-951e-c9db5fdc00b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:37.417670 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e934f3f6-8490-4b90-951e-c9db5fdc00b6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:37.447896 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:37.456468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1076/2155] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:37 2022] POST /identity/v3/projects => generated 381 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:37.460250 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e934f3f6-8490-4b90-951e-c9db5fdc00b6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:37.462721 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2730e975-2c73-4d4b-b971-58de6c8e373f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c28991387694c958cdd7a98e4ce12ae'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:37.463846 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2730e975-2c73-4d4b-b971-58de6c8e373f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c28991387694c958cdd7a98e4ce12ae'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:37.465335 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2730e975-2c73-4d4b-b971-58de6c8e373f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c28991387694c958cdd7a98e4ce12ae'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:37.480435 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2730e975-2c73-4d4b-b971-58de6c8e373f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c28991387694c958cdd7a98e4ce12ae'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:37.494126 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2730e975-2c73-4d4b-b971-58de6c8e373f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c28991387694c958cdd7a98e4ce12ae'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:37.518110 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1080/2156] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:37 2022] DELETE /identity/v3/users/8c28991387694c958cdd7a98e4ce12ae => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:05:37.533103 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e934f3f6-8490-4b90-951e-c9db5fdc00b6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:37.608303 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade501a-e203-4cf1-b2e1-606ec80c3c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:37.609004 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade501a-e203-4cf1-b2e1-606ec80c3c0a admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:37.609306 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade501a-e203-4cf1-b2e1-606ec80c3c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:37.609608 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade501a-e203-4cf1-b2e1-606ec80c3c0a admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:37.610605 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade501a-e203-4cf1-b2e1-606ec80c3c0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:37.612659 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade501a-e203-4cf1-b2e1-606ec80c3c0a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:37.638735 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1077/2157] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:37 2022] POST /identity/v3/users => generated 291 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:05:37.642998 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730e975-2c73-4d4b-b971-58de6c8e373f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:37.644471 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2730e975-2c73-4d4b-b971-58de6c8e373f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:37.680031 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730e975-2c73-4d4b-b971-58de6c8e373f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:37.687740 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27dc752-5652-4893-b71c-3c2bb87e0bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:37.688690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27dc752-5652-4893-b71c-3c2bb87e0bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:37.689099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27dc752-5652-4893-b71c-3c2bb87e0bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:37.689462 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27dc752-5652-4893-b71c-3c2bb87e0bd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:37.690338 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27dc752-5652-4893-b71c-3c2bb87e0bd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:37.693709 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27dc752-5652-4893-b71c-3c2bb87e0bd4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:37.712004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1081/2158] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:37.713213 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade501a-e203-4cf1-b2e1-606ec80c3c0a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:37.734991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade501a-e203-4cf1-b2e1-606ec80c3c0a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:37.744408 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63716a14-21e7-4265-a88c-ab66c4395615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:37.745106 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63716a14-21e7-4265-a88c-ab66c4395615 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:37.745400 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63716a14-21e7-4265-a88c-ab66c4395615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:37.745721 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63716a14-21e7-4265-a88c-ab66c4395615 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:37.746476 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63716a14-21e7-4265-a88c-ab66c4395615 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:37.751061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63716a14-21e7-4265-a88c-ab66c4395615 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:37.766548 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1078/2159] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:37.771712 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27dc752-5652-4893-b71c-3c2bb87e0bd4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:37.792241 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27dc752-5652-4893-b71c-3c2bb87e0bd4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:37.797721 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee9dedd-fdc4-4e03-88b5-9ca3a9bac029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:37.798699 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9dedd-fdc4-4e03-88b5-9ca3a9bac029 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:37.798980 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9dedd-fdc4-4e03-88b5-9ca3a9bac029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:37.799321 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9dedd-fdc4-4e03-88b5-9ca3a9bac029 admin admin] PATH_INFO: `/v3/projects/bb28d25de0df42c49f504c609f87efae/users/3dddd8bba6bc4bd588a772b2d5969ce1/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:37.834268 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63716a14-21e7-4265-a88c-ab66c4395615 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:37.838178 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-63716a14-21e7-4265-a88c-ab66c4395615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:37.838406 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee9dedd-fdc4-4e03-88b5-9ca3a9bac029 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb28d25de0df42c49f504c609f87efae, user_id=3dddd8bba6bc4bd588a772b2d5969ce1, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:37.845521 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee9dedd-fdc4-4e03-88b5-9ca3a9bac029 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:37.878659 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63716a14-21e7-4265-a88c-ab66c4395615 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:37.884147 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1082/2160] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:37 2022] PUT /identity/v3/projects/bb28d25de0df42c49f504c609f87efae/users/3dddd8bba6bc4bd588a772b2d5969ce1/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:37.886626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c462a-3c15-41a9-968a-d859fc79f770 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:37.886913 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c462a-3c15-41a9-968a-d859fc79f770 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:37.887195 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c462a-3c15-41a9-968a-d859fc79f770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:37.911258 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=77,keystone:DELETE=20,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:37.926006 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5b0c462a-3c15-41a9-968a-d859fc79f770 None None] MFA Rules not processed for user `3dddd8bba6bc4bd588a772b2d5969ce1`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:37.928999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5b0c462a-3c15-41a9-968a-d859fc79f770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:37.955767 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e11b7-53b6-4681-888a-361b539356e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:37.972239 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e11b7-53b6-4681-888a-361b539356e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:37.972836 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e11b7-53b6-4681-888a-361b539356e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:37.973530 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e11b7-53b6-4681-888a-361b539356e4 admin admin] PATH_INFO: `/v3/projects/f49a5f13c3ef42b8aa5a4f9a38ca9b76` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:37.976329 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e11b7-53b6-4681-888a-361b539356e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f49a5f13c3ef42b8aa5a4f9a38ca9b76)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:37.978729 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e11b7-53b6-4681-888a-361b539356e4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:38.034985 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:38.039311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1083/2161] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:38.044675 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c462a-3c15-41a9-968a-d859fc79f770 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:38.059093 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-523e11b7-53b6-4681-888a-361b539356e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f49a5f13c3ef42b8aa5a4f9a38ca9b76', 'request_id': 'req-523e11b7-53b6-4681-888a-361b539356e4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:38.067483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c462a-3c15-41a9-968a-d859fc79f770 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:38.069425 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-523e11b7-53b6-4681-888a-361b539356e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f49a5f13c3ef42b8aa5a4f9a38ca9b76', 'request_id': 'req-523e11b7-53b6-4681-888a-361b539356e4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:38.079457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-523e11b7-53b6-4681-888a-361b539356e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f49a5f13c3ef42b8aa5a4f9a38ca9b76', 'request_id': 'req-523e11b7-53b6-4681-888a-361b539356e4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:38.095915 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-523e11b7-53b6-4681-888a-361b539356e4 admin admin] The token cache is being invalidate because project f49a5f13c3ef42b8aa5a4f9a38ca9b76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:38.096331 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-523e11b7-53b6-4681-888a-361b539356e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:38.097779 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-523e11b7-53b6-4681-888a-361b539356e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:38.104754 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1079/2162] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:37 2022] DELETE /identity/v3/projects/f49a5f13c3ef42b8aa5a4f9a38ca9b76 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:38.106487 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e11b7-53b6-4681-888a-361b539356e4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:38.109147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-523e11b7-53b6-4681-888a-361b539356e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:38.159899 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320188c8-9a49-4d28-bf09-a08b3a6b4379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:38.160946 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320188c8-9a49-4d28-bf09-a08b3a6b4379 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:38.162015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320188c8-9a49-4d28-bf09-a08b3a6b4379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:38.164527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320188c8-9a49-4d28-bf09-a08b3a6b4379 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:38.165476 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320188c8-9a49-4d28-bf09-a08b3a6b4379 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:38.168219 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320188c8-9a49-4d28-bf09-a08b3a6b4379 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:38.213622 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1084/2163] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:38 2022] POST /identity/v3/projects => generated 381 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:38.215374 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320188c8-9a49-4d28-bf09-a08b3a6b4379 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:38.217165 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-320188c8-9a49-4d28-bf09-a08b3a6b4379 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:38.261085 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e11b7-53b6-4681-888a-361b539356e4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:38.342161 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1cdb1-1f69-43f0-8c1f-1b55122024aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:38.343678 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1cdb1-1f69-43f0-8c1f-1b55122024aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:38.344043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1cdb1-1f69-43f0-8c1f-1b55122024aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:38.344400 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1cdb1-1f69-43f0-8c1f-1b55122024aa admin admin] PATH_INFO: `/v3/users/95d1419f03104742a2bd3c56a840112e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:38.345759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a1cdb1-1f69-43f0-8c1f-1b55122024aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=95d1419f03104742a2bd3c56a840112e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:38.348850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a1cdb1-1f69-43f0-8c1f-1b55122024aa admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:38.360639 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320188c8-9a49-4d28-bf09-a08b3a6b4379 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:38.369185 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7222f-45bf-4cb6-93cc-d0bfa9d1ebf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:38.370421 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7222f-45bf-4cb6-93cc-d0bfa9d1ebf0 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:38.370421 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7222f-45bf-4cb6-93cc-d0bfa9d1ebf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:38.370947 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7222f-45bf-4cb6-93cc-d0bfa9d1ebf0 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:38.371740 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7222f-45bf-4cb6-93cc-d0bfa9d1ebf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:38.373773 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7222f-45bf-4cb6-93cc-d0bfa9d1ebf0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:38.401469 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1085/2164] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:38 2022] POST /identity/v3/users => generated 297 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:38.401469 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7222f-45bf-4cb6-93cc-d0bfa9d1ebf0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:38.402380 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-42c7222f-45bf-4cb6-93cc-d0bfa9d1ebf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:38.408523 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-37a1cdb1-1f69-43f0-8c1f-1b55122024aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95d1419f03104742a2bd3c56a840112e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:38.409638 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-37a1cdb1-1f69-43f0-8c1f-1b55122024aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95d1419f03104742a2bd3c56a840112e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:38.410196 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-37a1cdb1-1f69-43f0-8c1f-1b55122024aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95d1419f03104742a2bd3c56a840112e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:38.417459 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7222f-45bf-4cb6-93cc-d0bfa9d1ebf0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:38.423433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59137e9b-4250-4d88-9153-3c03b00a1de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:38.424702 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59137e9b-4250-4d88-9153-3c03b00a1de4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:38.425517 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59137e9b-4250-4d88-9153-3c03b00a1de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:38.425900 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59137e9b-4250-4d88-9153-3c03b00a1de4 admin admin] PATH_INFO: `/v3/projects/88f6ce0aa585487a859cb31986a80fdb` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:38.427790 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-37a1cdb1-1f69-43f0-8c1f-1b55122024aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95d1419f03104742a2bd3c56a840112e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:38.427949 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59137e9b-4250-4d88-9153-3c03b00a1de4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88f6ce0aa585487a859cb31986a80fdb)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:38.429800 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59137e9b-4250-4d88-9153-3c03b00a1de4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:38.439193 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-37a1cdb1-1f69-43f0-8c1f-1b55122024aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95d1419f03104742a2bd3c56a840112e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:38.441633 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:38.449211 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:38.467967 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1080/2165] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:38 2022] DELETE /identity/v3/users/95d1419f03104742a2bd3c56a840112e => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:38.469489 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1cdb1-1f69-43f0-8c1f-1b55122024aa admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:38.471227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-37a1cdb1-1f69-43f0-8c1f-1b55122024aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:38.499395 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-59137e9b-4250-4d88-9153-3c03b00a1de4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88f6ce0aa585487a859cb31986a80fdb', 'request_id': 'req-59137e9b-4250-4d88-9153-3c03b00a1de4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:38.509834 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-59137e9b-4250-4d88-9153-3c03b00a1de4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88f6ce0aa585487a859cb31986a80fdb', 'request_id': 'req-59137e9b-4250-4d88-9153-3c03b00a1de4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:38.518881 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-59137e9b-4250-4d88-9153-3c03b00a1de4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88f6ce0aa585487a859cb31986a80fdb', 'request_id': 'req-59137e9b-4250-4d88-9153-3c03b00a1de4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:38.538386 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-59137e9b-4250-4d88-9153-3c03b00a1de4 admin admin] The token cache is being invalidate because project 88f6ce0aa585487a859cb31986a80fdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:38.538675 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-59137e9b-4250-4d88-9153-3c03b00a1de4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:38.539533 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-59137e9b-4250-4d88-9153-3c03b00a1de4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:38.541311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1086/2166] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:38 2022] DELETE /identity/v3/projects/88f6ce0aa585487a859cb31986a80fdb => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:38.547159 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6878b-ffe8-4e40-b5ba-9d613dabf7dd None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:38.547526 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6878b-ffe8-4e40-b5ba-9d613dabf7dd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:38.548288 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6878b-ffe8-4e40-b5ba-9d613dabf7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:38.573045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ffb6878b-ffe8-4e40-b5ba-9d613dabf7dd None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:38.574234 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ffb6878b-ffe8-4e40-b5ba-9d613dabf7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:38.576773 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1cdb1-1f69-43f0-8c1f-1b55122024aa admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:38.659698 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1087/2167] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:05:38.666855 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7634558-32ce-4858-9aa5-1c8426b77459 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:38.666855 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7634558-32ce-4858-9aa5-1c8426b77459 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:38.667312 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7634558-32ce-4858-9aa5-1c8426b77459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:38.671807 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e1d61f-02c1-4395-89de-86a34053e7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:38.672490 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1d61f-02c1-4395-89de-86a34053e7ac admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:38.672780 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1d61f-02c1-4395-89de-86a34053e7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:38.673065 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1d61f-02c1-4395-89de-86a34053e7ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:38.673860 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e1d61f-02c1-4395-89de-86a34053e7ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:38.675647 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e1d61f-02c1-4395-89de-86a34053e7ac admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:38.687108 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1081/2168] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:38.691329 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e1d61f-02c1-4395-89de-86a34053e7ac admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:38.693413 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-55e1d61f-02c1-4395-89de-86a34053e7ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:38.702288 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c7634558-32ce-4858-9aa5-1c8426b77459 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:38.704042 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c7634558-32ce-4858-9aa5-1c8426b77459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:38.712760 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e1d61f-02c1-4395-89de-86a34053e7ac admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:38.718049 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec59fe40-bc39-4a53-b571-e4643495684d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:38.718708 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1088/2169] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:05:38.722064 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59fe40-bc39-4a53-b571-e4643495684d admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:38.722361 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59fe40-bc39-4a53-b571-e4643495684d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:38.723416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59fe40-bc39-4a53-b571-e4643495684d admin admin] PATH_INFO: `/v3/projects/5ab753a4620346028af6fd5701c1da27/users/6962f8afe6f9496c95a345132ed9d23b/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:38.725220 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7634558-32ce-4858-9aa5-1c8426b77459 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:38.744109 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec59fe40-bc39-4a53-b571-e4643495684d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ab753a4620346028af6fd5701c1da27, user_id=6962f8afe6f9496c95a345132ed9d23b, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:38.747515 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec59fe40-bc39-4a53-b571-e4643495684d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:38.761056 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7634558-32ce-4858-9aa5-1c8426b77459 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:38.773893 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1082/2170] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:38 2022] PUT /identity/v3/projects/5ab753a4620346028af6fd5701c1da27/users/6962f8afe6f9496c95a345132ed9d23b/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:38.775164 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec59fe40-bc39-4a53-b571-e4643495684d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:38.826736 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40520995-007f-4ffc-8946-d3004eb498cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:38.827773 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40520995-007f-4ffc-8946-d3004eb498cf admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:38.828280 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40520995-007f-4ffc-8946-d3004eb498cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:38.828734 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40520995-007f-4ffc-8946-d3004eb498cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:38.829836 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40520995-007f-4ffc-8946-d3004eb498cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:05:38.830238 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40520995-007f-4ffc-8946-d3004eb498cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:38.831992 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40520995-007f-4ffc-8946-d3004eb498cf admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:38.851324 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1089/2171] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 16:05:38.852288 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40520995-007f-4ffc-8946-d3004eb498cf admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:38.855459 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-40520995-007f-4ffc-8946-d3004eb498cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:38.886455 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40520995-007f-4ffc-8946-d3004eb498cf admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:38.893248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0189f5f1-f215-45f0-88a6-c8c478607228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:38.894386 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189f5f1-f215-45f0-88a6-c8c478607228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:38.894673 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189f5f1-f215-45f0-88a6-c8c478607228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:38.895378 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189f5f1-f215-45f0-88a6-c8c478607228 admin admin] PATH_INFO: `/v3/projects/f66a04ebde8a4e628b700eee4ea2448f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:38.897282 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0189f5f1-f215-45f0-88a6-c8c478607228 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f66a04ebde8a4e628b700eee4ea2448f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:38.899947 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0189f5f1-f215-45f0-88a6-c8c478607228 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:38.932893 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec59fe40-bc39-4a53-b571-e4643495684d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:38.945899 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:38.949089 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b73ce0-ecca-4769-8f41-c80795161bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:38.955215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b73ce0-ecca-4769-8f41-c80795161bd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:38.955533 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b73ce0-ecca-4769-8f41-c80795161bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:38.955825 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b73ce0-ecca-4769-8f41-c80795161bd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:38.957907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b73ce0-ecca-4769-8f41-c80795161bd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:38.961424 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b73ce0-ecca-4769-8f41-c80795161bd1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:38.972378 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:39.002179 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1083/2172] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:39.008450 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b73ce0-ecca-4769-8f41-c80795161bd1 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:39.022703 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b73ce0-ecca-4769-8f41-c80795161bd1 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:39.029598 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0189f5f1-f215-45f0-88a6-c8c478607228 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f66a04ebde8a4e628b700eee4ea2448f', 'request_id': 'req-0189f5f1-f215-45f0-88a6-c8c478607228'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:39.046612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0189f5f1-f215-45f0-88a6-c8c478607228 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f66a04ebde8a4e628b700eee4ea2448f', 'request_id': 'req-0189f5f1-f215-45f0-88a6-c8c478607228'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:39.054960 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0189f5f1-f215-45f0-88a6-c8c478607228 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f66a04ebde8a4e628b700eee4ea2448f', 'request_id': 'req-0189f5f1-f215-45f0-88a6-c8c478607228'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:39.079728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0189f5f1-f215-45f0-88a6-c8c478607228 admin admin] The token cache is being invalidate because project f66a04ebde8a4e628b700eee4ea2448f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:39.080031 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0189f5f1-f215-45f0-88a6-c8c478607228 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:39.081010 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0189f5f1-f215-45f0-88a6-c8c478607228 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:39.087463 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1090/2173] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:38 2022] DELETE /identity/v3/projects/f66a04ebde8a4e628b700eee4ea2448f => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:39.087463 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0189f5f1-f215-45f0-88a6-c8c478607228 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:39.087463 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0189f5f1-f215-45f0-88a6-c8c478607228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:39.144852 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4001d62-c6ca-4880-a208-fdbd870c6dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:39.145659 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4001d62-c6ca-4880-a208-fdbd870c6dde admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:39.146057 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4001d62-c6ca-4880-a208-fdbd870c6dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:39.146409 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4001d62-c6ca-4880-a208-fdbd870c6dde admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:39.147761 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4001d62-c6ca-4880-a208-fdbd870c6dde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:39.149795 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4001d62-c6ca-4880-a208-fdbd870c6dde admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:39.194334 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1084/2174] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:39 2022] POST /identity/v3/projects => generated 383 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:39.196189 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228af8b5-aeae-44e5-b5a9-c8236ecc435d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:39.197489 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228af8b5-aeae-44e5-b5a9-c8236ecc435d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:39.197489 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228af8b5-aeae-44e5-b5a9-c8236ecc435d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:39.206137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0189f5f1-f215-45f0-88a6-c8c478607228 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:39.222126 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-228af8b5-aeae-44e5-b5a9-c8236ecc435d None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:39.226278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-228af8b5-aeae-44e5-b5a9-c8236ecc435d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:39.289750 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce111d6-daed-44a7-9a4d-49786676f6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:39.290645 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce111d6-daed-44a7-9a4d-49786676f6f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:39.290929 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce111d6-daed-44a7-9a4d-49786676f6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:39.291221 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce111d6-daed-44a7-9a4d-49786676f6f2 admin admin] PATH_INFO: `/v3/projects/5ab753a4620346028af6fd5701c1da27/users/6962f8afe6f9496c95a345132ed9d23b/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:39.293249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce111d6-daed-44a7-9a4d-49786676f6f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ab753a4620346028af6fd5701c1da27, user_id=6962f8afe6f9496c95a345132ed9d23b, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:39.296645 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce111d6-daed-44a7-9a4d-49786676f6f2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:39.304141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1085/2175] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:39.310835 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228af8b5-aeae-44e5-b5a9-c8236ecc435d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:39.312757 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-228af8b5-aeae-44e5-b5a9-c8236ecc435d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:39.327394 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1091/2176] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:39 2022] PUT /identity/v3/projects/5ab753a4620346028af6fd5701c1da27/users/6962f8afe6f9496c95a345132ed9d23b/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:39.327394 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e51cc4-6e56-4ada-ab9d-0727a0938425 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:39.327394 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e51cc4-6e56-4ada-ab9d-0727a0938425 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:39.327394 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e51cc4-6e56-4ada-ab9d-0727a0938425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:39.327836 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228af8b5-aeae-44e5-b5a9-c8236ecc435d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:39.348749 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-45e51cc4-6e56-4ada-ab9d-0727a0938425 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:39.350147 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-45e51cc4-6e56-4ada-ab9d-0727a0938425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:39.390759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6abae44-71fe-4fc8-be3c-ab011c6a6214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:39.391455 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abae44-71fe-4fc8-be3c-ab011c6a6214 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:39.391747 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abae44-71fe-4fc8-be3c-ab011c6a6214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:39.392028 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abae44-71fe-4fc8-be3c-ab011c6a6214 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:39.392746 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6abae44-71fe-4fc8-be3c-ab011c6a6214 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:39.394745 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6abae44-71fe-4fc8-be3c-ab011c6a6214 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:39.405686 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:39.419544 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1086/2177] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:39 2022] POST /identity/v3/users => generated 292 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:39.423896 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15539c5-3337-419c-ac6f-abea417db58a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:39.424190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15539c5-3337-419c-ac6f-abea417db58a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:39.424469 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15539c5-3337-419c-ac6f-abea417db58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:39.426624 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1092/2178] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:39.430311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e51cc4-6e56-4ada-ab9d-0727a0938425 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:39.439864 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e51cc4-6e56-4ada-ab9d-0727a0938425 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:39.445297 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af65a237-f6db-4098-af75-233b72fd8f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:39.446002 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af65a237-f6db-4098-af75-233b72fd8f8a admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:39.446286 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af65a237-f6db-4098-af75-233b72fd8f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:39.446563 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af65a237-f6db-4098-af75-233b72fd8f8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:39.447306 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af65a237-f6db-4098-af75-233b72fd8f8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:39.448995 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af65a237-f6db-4098-af75-233b72fd8f8a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:39.455842 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:39.459786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d15539c5-3337-419c-ac6f-abea417db58a None None] MFA Rules not processed for user `6962f8afe6f9496c95a345132ed9d23b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:39.461445 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1093/2179] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:39.462333 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af65a237-f6db-4098-af75-233b72fd8f8a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:39.464779 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d15539c5-3337-419c-ac6f-abea417db58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:39.473967 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af65a237-f6db-4098-af75-233b72fd8f8a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:39.479840 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a717f-e726-44e9-b3d4-d4b6c6985ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:39.480526 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a717f-e726-44e9-b3d4-d4b6c6985ba3 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:39.481071 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a717f-e726-44e9-b3d4-d4b6c6985ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:39.481250 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a717f-e726-44e9-b3d4-d4b6c6985ba3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:39.483409 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a717f-e726-44e9-b3d4-d4b6c6985ba3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:05:39.483743 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a717f-e726-44e9-b3d4-d4b6c6985ba3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:39.485452 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a717f-e726-44e9-b3d4-d4b6c6985ba3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:39.502414 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1094/2180] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:05:39.503708 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a717f-e726-44e9-b3d4-d4b6c6985ba3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:39.516498 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a717f-e726-44e9-b3d4-d4b6c6985ba3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:39.522623 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd7107-79f5-48a5-88e6-e3b0467d5b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:39.523616 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd7107-79f5-48a5-88e6-e3b0467d5b93 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:39.524184 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd7107-79f5-48a5-88e6-e3b0467d5b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:39.524936 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd7107-79f5-48a5-88e6-e3b0467d5b93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:39.526067 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd7107-79f5-48a5-88e6-e3b0467d5b93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:39.528712 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd7107-79f5-48a5-88e6-e3b0467d5b93 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:39.545047 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1095/2181] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:39.546924 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd7107-79f5-48a5-88e6-e3b0467d5b93 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:39.565980 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd7107-79f5-48a5-88e6-e3b0467d5b93 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:39.589048 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3740fb71-aeb3-4025-8d5b-439f9a5aa30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:39.590131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740fb71-aeb3-4025-8d5b-439f9a5aa30a admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:39.591354 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740fb71-aeb3-4025-8d5b-439f9a5aa30a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:39.591748 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740fb71-aeb3-4025-8d5b-439f9a5aa30a admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:39.592923 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3740fb71-aeb3-4025-8d5b-439f9a5aa30a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:39.594246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1087/2182] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:39 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:39.594939 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15539c5-3337-419c-ac6f-abea417db58a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:39.595664 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3740fb71-aeb3-4025-8d5b-439f9a5aa30a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:39.621435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15539c5-3337-419c-ac6f-abea417db58a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:39.626511 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b5341-dbe1-4736-acb8-5f780bbfe862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:39.627783 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b5341-dbe1-4736-acb8-5f780bbfe862 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:39.628122 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b5341-dbe1-4736-acb8-5f780bbfe862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:39.628483 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b5341-dbe1-4736-acb8-5f780bbfe862 admin admin] PATH_INFO: `/v3/projects/b782b3c15e134b2bab26628b2286f96d/users/147f52c566104bffb04e9d12e4179c13/roles/54195e660fc349978b39d5794765a095` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:39.640514 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1096/2183] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:39 2022] POST /identity/v3/projects => generated 359 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:39.642027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3740fb71-aeb3-4025-8d5b-439f9a5aa30a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:39.644821 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3740fb71-aeb3-4025-8d5b-439f9a5aa30a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:39.648379 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b5341-dbe1-4736-acb8-5f780bbfe862 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b782b3c15e134b2bab26628b2286f96d, user_id=147f52c566104bffb04e9d12e4179c13, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:39.655185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b5341-dbe1-4736-acb8-5f780bbfe862 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:39.682266 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1088/2184] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:39 2022] PUT /identity/v3/projects/b782b3c15e134b2bab26628b2286f96d/users/147f52c566104bffb04e9d12e4179c13/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:39.694520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b5341-dbe1-4736-acb8-5f780bbfe862 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:39.723175 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3740fb71-aeb3-4025-8d5b-439f9a5aa30a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:39.778458 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b5341-dbe1-4736-acb8-5f780bbfe862 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:39.779312 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153b71f-684b-45ad-9cf8-9966f44182bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:39.780378 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153b71f-684b-45ad-9cf8-9966f44182bb service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:39.780927 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153b71f-684b-45ad-9cf8-9966f44182bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:39.781467 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153b71f-684b-45ad-9cf8-9966f44182bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:39.783226 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae58d41-ab71-42e6-b10e-20509522c420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:39.783990 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae58d41-ab71-42e6-b10e-20509522c420 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:39.784273 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae58d41-ab71-42e6-b10e-20509522c420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:39.784564 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae58d41-ab71-42e6-b10e-20509522c420 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:39.785318 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae58d41-ab71-42e6-b10e-20509522c420 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:39.787838 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae58d41-ab71-42e6-b10e-20509522c420 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:39.811612 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1089/2185] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:39 2022] POST /identity/v3/users => generated 280 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:39.812903 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb90ff1-ddcf-4a18-ae88-cbd428488f08 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:39.813198 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb90ff1-ddcf-4a18-ae88-cbd428488f08 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:39.813481 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb90ff1-ddcf-4a18-ae88-cbd428488f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:39.842919 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-bdb90ff1-ddcf-4a18-ae88-cbd428488f08 None None] MFA Rules not processed for user `147f52c566104bffb04e9d12e4179c13`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:39.844219 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bdb90ff1-ddcf-4a18-ae88-cbd428488f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:39.848774 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f153b71f-684b-45ad-9cf8-9966f44182bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:39.850812 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f153b71f-684b-45ad-9cf8-9966f44182bb service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:39.855920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:39.862114 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1097/2186] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:39 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:39.863761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153b71f-684b-45ad-9cf8-9966f44182bb service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:39.880706 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153b71f-684b-45ad-9cf8-9966f44182bb service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:39.886988 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ee3249-1975-4d67-ab19-a42a12884fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:39.887789 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee3249-1975-4d67-ab19-a42a12884fd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:39.887937 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee3249-1975-4d67-ab19-a42a12884fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:39.888233 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee3249-1975-4d67-ab19-a42a12884fd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:39.889439 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ee3249-1975-4d67-ab19-a42a12884fd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:39.891217 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ee3249-1975-4d67-ab19-a42a12884fd6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:39.903530 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1098/2187] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:39.907737 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ee3249-1975-4d67-ab19-a42a12884fd6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:39.915373 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:39.923706 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ee3249-1975-4d67-ab19-a42a12884fd6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:39.930496 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ba7e5-ed79-454d-b87c-c305ffc68bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:39.931173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ba7e5-ed79-454d-b87c-c305ffc68bdb admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:39.931462 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ba7e5-ed79-454d-b87c-c305ffc68bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:39.931743 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ba7e5-ed79-454d-b87c-c305ffc68bdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:39.932480 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811ba7e5-ed79-454d-b87c-c305ffc68bdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:39.935142 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811ba7e5-ed79-454d-b87c-c305ffc68bdb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:39.942036 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1090/2188] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:39.947918 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1099/2189] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 16:05:39.951717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb90ff1-ddcf-4a18-ae88-cbd428488f08 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:39.964081 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb90ff1-ddcf-4a18-ae88-cbd428488f08 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:39.970621 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a97ea19-e04e-41b3-9d8e-d05401c46913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:39.970750 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ba7e5-ed79-454d-b87c-c305ffc68bdb admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:39.978975 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97ea19-e04e-41b3-9d8e-d05401c46913 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:39.978975 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97ea19-e04e-41b3-9d8e-d05401c46913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:39.978975 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97ea19-e04e-41b3-9d8e-d05401c46913 admin admin] PATH_INFO: `/v3/projects/16f912f8487447eb837d640589dae79e/users/cdf53b24b292492dbcb8a183446fb58a/roles/54195e660fc349978b39d5794765a095` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:39.988320 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ba7e5-ed79-454d-b87c-c305ffc68bdb admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:39.994042 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd74f1f-0e06-4c80-97c8-24d2fd5220ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:39.994694 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd74f1f-0e06-4c80-97c8-24d2fd5220ea service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:39.995506 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd74f1f-0e06-4c80-97c8-24d2fd5220ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:39.995506 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd74f1f-0e06-4c80-97c8-24d2fd5220ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:40.007394 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a97ea19-e04e-41b3-9d8e-d05401c46913 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16f912f8487447eb837d640589dae79e, user_id=cdf53b24b292492dbcb8a183446fb58a, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:40.010158 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a97ea19-e04e-41b3-9d8e-d05401c46913 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:40.010846 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd74f1f-0e06-4c80-97c8-24d2fd5220ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:40.012456 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd74f1f-0e06-4c80-97c8-24d2fd5220ea service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:40.023479 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1100/2190] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:40.024146 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd74f1f-0e06-4c80-97c8-24d2fd5220ea service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:40.025623 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2bd74f1f-0e06-4c80-97c8-24d2fd5220ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:40.032917 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1091/2191] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:39 2022] PUT /identity/v3/projects/16f912f8487447eb837d640589dae79e/users/cdf53b24b292492dbcb8a183446fb58a/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:05:40.038904 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd74f1f-0e06-4c80-97c8-24d2fd5220ea service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:40.043805 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89796ee-a2f7-4320-9900-b6c30a245bda None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:40.044280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89796ee-a2f7-4320-9900-b6c30a245bda None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:40.044610 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89796ee-a2f7-4320-9900-b6c30a245bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:40.102460 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e89796ee-a2f7-4320-9900-b6c30a245bda None None] MFA Rules not processed for user `cdf53b24b292492dbcb8a183446fb58a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:40.103603 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e89796ee-a2f7-4320-9900-b6c30a245bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:40.109522 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3556a89d-6748-418f-96c5-af23b2c47540 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:40.110211 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556a89d-6748-418f-96c5-af23b2c47540 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:40.110484 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556a89d-6748-418f-96c5-af23b2c47540 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:40.110764 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556a89d-6748-418f-96c5-af23b2c47540 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:40.112703 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3556a89d-6748-418f-96c5-af23b2c47540 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:40.203553 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1092/2192] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:40 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:40.240008 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3556a89d-6748-418f-96c5-af23b2c47540 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:40.242938 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3556a89d-6748-418f-96c5-af23b2c47540 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:40.255039 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1101/2193] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:40.475550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89796ee-a2f7-4320-9900-b6c30a245bda None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:40.487829 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89796ee-a2f7-4320-9900-b6c30a245bda None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:40.492902 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5da0e-b80e-4fe4-809b-971f870ec393 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:40.494183 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5da0e-b80e-4fe4-809b-971f870ec393 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:40.494183 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5da0e-b80e-4fe4-809b-971f870ec393 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:40.500035 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5da0e-b80e-4fe4-809b-971f870ec393 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:40.528454 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc5da0e-b80e-4fe4-809b-971f870ec393 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:40.533994 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc5da0e-b80e-4fe4-809b-971f870ec393 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:40.549887 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1093/2194] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:40 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:40.570583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3556a89d-6748-418f-96c5-af23b2c47540 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:40.692041 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3556a89d-6748-418f-96c5-af23b2c47540 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:40.701404 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d53e95c-c929-4093-930e-9159bb20351a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:40.702386 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53e95c-c929-4093-930e-9159bb20351a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:40.702683 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53e95c-c929-4093-930e-9159bb20351a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:40.702965 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53e95c-c929-4093-930e-9159bb20351a admin admin] PATH_INFO: `/v3/users/cdf53b24b292492dbcb8a183446fb58a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:40.704223 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d53e95c-c929-4093-930e-9159bb20351a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdf53b24b292492dbcb8a183446fb58a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:40.706270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d53e95c-c929-4093-930e-9159bb20351a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:40.747799 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:40.795488 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2d53e95c-c929-4093-930e-9159bb20351a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdf53b24b292492dbcb8a183446fb58a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:40.798215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2d53e95c-c929-4093-930e-9159bb20351a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdf53b24b292492dbcb8a183446fb58a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:40.800414 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2d53e95c-c929-4093-930e-9159bb20351a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdf53b24b292492dbcb8a183446fb58a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:40.814953 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2d53e95c-c929-4093-930e-9159bb20351a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdf53b24b292492dbcb8a183446fb58a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:40.826305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2d53e95c-c929-4093-930e-9159bb20351a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdf53b24b292492dbcb8a183446fb58a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:40.876502 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1102/2195] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:40 2022] DELETE /identity/v3/users/cdf53b24b292492dbcb8a183446fb58a => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:40.881440 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628f44f-78a4-49b6-ab91-33c1ba934e60 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:40.882037 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628f44f-78a4-49b6-ab91-33c1ba934e60 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:40.882037 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628f44f-78a4-49b6-ab91-33c1ba934e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:40.908413 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8628f44f-78a4-49b6-ab91-33c1ba934e60 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:40.909955 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8628f44f-78a4-49b6-ab91-33c1ba934e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:41.001488 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1094/2196] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:05:41.019788 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d53e95c-c929-4093-930e-9159bb20351a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:41.022533 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2d53e95c-c929-4093-930e-9159bb20351a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:41.091222 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d53e95c-c929-4093-930e-9159bb20351a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:41.094913 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee8fe5-c8c1-4f39-93bd-6ab8fe926881 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:41.095782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee8fe5-c8c1-4f39-93bd-6ab8fe926881 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:41.095897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee8fe5-c8c1-4f39-93bd-6ab8fe926881 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:41.096094 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee8fe5-c8c1-4f39-93bd-6ab8fe926881 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:41.107111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee8fe5-c8c1-4f39-93bd-6ab8fe926881 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:41.108640 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee8fe5-c8c1-4f39-93bd-6ab8fe926881 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:41.118904 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1103/2197] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:41.306307 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628f44f-78a4-49b6-ab91-33c1ba934e60 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:41.307772 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8628f44f-78a4-49b6-ab91-33c1ba934e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:41.320687 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628f44f-78a4-49b6-ab91-33c1ba934e60 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:41.326477 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2051a4-d424-4f42-9f2d-e0226750c217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:41.327362 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2051a4-d424-4f42-9f2d-e0226750c217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:41.327649 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2051a4-d424-4f42-9f2d-e0226750c217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:41.327929 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2051a4-d424-4f42-9f2d-e0226750c217 admin admin] PATH_INFO: `/v3/users/05afc537455c4a79be9fc8c4fdb1fc96` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:41.329139 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2051a4-d424-4f42-9f2d-e0226750c217 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05afc537455c4a79be9fc8c4fdb1fc96)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:41.331372 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2051a4-d424-4f42-9f2d-e0226750c217 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:41.425840 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:41.432446 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-aa2051a4-d424-4f42-9f2d-e0226750c217 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05afc537455c4a79be9fc8c4fdb1fc96'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:41.433253 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-aa2051a4-d424-4f42-9f2d-e0226750c217 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05afc537455c4a79be9fc8c4fdb1fc96'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:41.434167 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-aa2051a4-d424-4f42-9f2d-e0226750c217 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05afc537455c4a79be9fc8c4fdb1fc96'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:41.443192 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-aa2051a4-d424-4f42-9f2d-e0226750c217 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05afc537455c4a79be9fc8c4fdb1fc96'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:41.459260 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-aa2051a4-d424-4f42-9f2d-e0226750c217 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05afc537455c4a79be9fc8c4fdb1fc96'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:41.491383 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1095/2198] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:41 2022] DELETE /identity/v3/users/05afc537455c4a79be9fc8c4fdb1fc96 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:41.648939 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee8fe5-c8c1-4f39-93bd-6ab8fe926881 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:41.650728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-89ee8fe5-c8c1-4f39-93bd-6ab8fe926881 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:41.665224 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee8fe5-c8c1-4f39-93bd-6ab8fe926881 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:41.746235 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd8754-9f84-4edb-bbf0-e7700d78cc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:41.747089 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd8754-9f84-4edb-bbf0-e7700d78cc02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:41.747372 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd8754-9f84-4edb-bbf0-e7700d78cc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:41.747650 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd8754-9f84-4edb-bbf0-e7700d78cc02 admin admin] PATH_INFO: `/v3/projects/16f912f8487447eb837d640589dae79e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:41.748838 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd8754-9f84-4edb-bbf0-e7700d78cc02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16f912f8487447eb837d640589dae79e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:41.750792 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd8754-9f84-4edb-bbf0-e7700d78cc02 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:41.777412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2051a4-d424-4f42-9f2d-e0226750c217 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:41.778895 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-aa2051a4-d424-4f42-9f2d-e0226750c217 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:41.793433 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2051a4-d424-4f42-9f2d-e0226750c217 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:41.798235 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2352ff1-6f63-43fd-9790-30ba01cd63fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:41.799182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2352ff1-6f63-43fd-9790-30ba01cd63fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:41.799533 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2352ff1-6f63-43fd-9790-30ba01cd63fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:41.799879 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2352ff1-6f63-43fd-9790-30ba01cd63fa admin admin] PATH_INFO: `/v3/projects/c876b26300114f33b915345c7dc524e2` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:41.801168 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2352ff1-6f63-43fd-9790-30ba01cd63fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=c876b26300114f33b915345c7dc524e2)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:41.803852 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2352ff1-6f63-43fd-9790-30ba01cd63fa admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:41.820857 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:41.828839 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-82cd8754-9f84-4edb-bbf0-e7700d78cc02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16f912f8487447eb837d640589dae79e', 'request_id': 'req-82cd8754-9f84-4edb-bbf0-e7700d78cc02'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:41.844456 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-82cd8754-9f84-4edb-bbf0-e7700d78cc02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16f912f8487447eb837d640589dae79e', 'request_id': 'req-82cd8754-9f84-4edb-bbf0-e7700d78cc02'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:41.852960 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-82cd8754-9f84-4edb-bbf0-e7700d78cc02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16f912f8487447eb837d640589dae79e', 'request_id': 'req-82cd8754-9f84-4edb-bbf0-e7700d78cc02'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:41.867096 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f2352ff1-6f63-43fd-9790-30ba01cd63fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c876b26300114f33b915345c7dc524e2', 'request_id': 'req-f2352ff1-6f63-43fd-9790-30ba01cd63fa'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:41.867641 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-82cd8754-9f84-4edb-bbf0-e7700d78cc02 admin admin] The token cache is being invalidate because project 16f912f8487447eb837d640589dae79e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:41.867979 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-82cd8754-9f84-4edb-bbf0-e7700d78cc02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:41.868781 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-82cd8754-9f84-4edb-bbf0-e7700d78cc02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:41.870954 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1104/2199] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:41 2022] DELETE /identity/v3/projects/16f912f8487447eb837d640589dae79e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:41.875814 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f2352ff1-6f63-43fd-9790-30ba01cd63fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c876b26300114f33b915345c7dc524e2', 'request_id': 'req-f2352ff1-6f63-43fd-9790-30ba01cd63fa'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:41.876145 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37567a-924d-4ce7-8f65-0a1c0a8a3be2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:41.877165 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37567a-924d-4ce7-8f65-0a1c0a8a3be2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:41.877529 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37567a-924d-4ce7-8f65-0a1c0a8a3be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:41.885376 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f2352ff1-6f63-43fd-9790-30ba01cd63fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c876b26300114f33b915345c7dc524e2', 'request_id': 'req-f2352ff1-6f63-43fd-9790-30ba01cd63fa'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:41.897253 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f2352ff1-6f63-43fd-9790-30ba01cd63fa admin admin] The token cache is being invalidate because project c876b26300114f33b915345c7dc524e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:41.897816 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f2352ff1-6f63-43fd-9790-30ba01cd63fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:41.898515 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f2352ff1-6f63-43fd-9790-30ba01cd63fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:41.900066 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1096/2200] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:41 2022] DELETE /identity/v3/projects/c876b26300114f33b915345c7dc524e2 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:05:41.902673 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0c37567a-924d-4ce7-8f65-0a1c0a8a3be2 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:41.903934 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0c37567a-924d-4ce7-8f65-0a1c0a8a3be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:41.905143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9eea5d-ff9f-431b-aac1-8f04d691ce61 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:41.905501 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9eea5d-ff9f-431b-aac1-8f04d691ce61 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:41.905878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9eea5d-ff9f-431b-aac1-8f04d691ce61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:41.933937 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3d9eea5d-ff9f-431b-aac1-8f04d691ce61 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:41.935537 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3d9eea5d-ff9f-431b-aac1-8f04d691ce61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:41.987038 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1105/2201] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:41.995288 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f218ebb-a4eb-4e61-83fb-58b628376210 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:41.995574 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f218ebb-a4eb-4e61-83fb-58b628376210 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:41.995850 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f218ebb-a4eb-4e61-83fb-58b628376210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:42.009223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1097/2202] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:42.016735 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1f218ebb-a4eb-4e61-83fb-58b628376210 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:42.017563 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd578f0d-160e-4518-96a8-a5c705bd36a7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:42.017896 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd578f0d-160e-4518-96a8-a5c705bd36a7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:42.018175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd578f0d-160e-4518-96a8-a5c705bd36a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:42.018950 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1f218ebb-a4eb-4e61-83fb-58b628376210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:42.032037 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1106/2203] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:42.036267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f218ebb-a4eb-4e61-83fb-58b628376210 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:42.042914 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-fd578f0d-160e-4518-96a8-a5c705bd36a7 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:42.044238 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fd578f0d-160e-4518-96a8-a5c705bd36a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:42.045854 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f218ebb-a4eb-4e61-83fb-58b628376210 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:42.051386 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7250fd-78d5-4134-8c72-a6afc5a8f8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:42.052077 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7250fd-78d5-4134-8c72-a6afc5a8f8ad admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:42.052396 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7250fd-78d5-4134-8c72-a6afc5a8f8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:42.052709 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7250fd-78d5-4134-8c72-a6afc5a8f8ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:42.054152 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7250fd-78d5-4134-8c72-a6afc5a8f8ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:05:42.054536 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1098/2204] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:05:42.054744 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7250fd-78d5-4134-8c72-a6afc5a8f8ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:42.056441 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7250fd-78d5-4134-8c72-a6afc5a8f8ad admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:42.058744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd578f0d-160e-4518-96a8-a5c705bd36a7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:42.068069 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd578f0d-160e-4518-96a8-a5c705bd36a7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:42.073783 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1107/2205] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:05:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 16:05:42.074786 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7250fd-78d5-4134-8c72-a6afc5a8f8ad admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:42.075955 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce214a71-0940-4174-86dc-d477461d81c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:42.076806 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce214a71-0940-4174-86dc-d477461d81c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:42.077014 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce214a71-0940-4174-86dc-d477461d81c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:42.077301 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce214a71-0940-4174-86dc-d477461d81c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:42.078131 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce214a71-0940-4174-86dc-d477461d81c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:05:42.078442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce214a71-0940-4174-86dc-d477461d81c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:42.080296 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce214a71-0940-4174-86dc-d477461d81c9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:42.084738 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7250fd-78d5-4134-8c72-a6afc5a8f8ad admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:42.089278 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41912ca8-9750-4d16-9c3f-cddbec963271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:42.089944 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41912ca8-9750-4d16-9c3f-cddbec963271 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:42.090228 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41912ca8-9750-4d16-9c3f-cddbec963271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:42.090499 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41912ca8-9750-4d16-9c3f-cddbec963271 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:42.091175 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41912ca8-9750-4d16-9c3f-cddbec963271 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:42.093054 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41912ca8-9750-4d16-9c3f-cddbec963271 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:42.097194 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1099/2206] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:05:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:05:42.098458 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce214a71-0940-4174-86dc-d477461d81c9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:42.109382 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce214a71-0940-4174-86dc-d477461d81c9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:42.114843 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c976bb9-11eb-451a-be2d-c2ebc9c13d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:42.116210 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c976bb9-11eb-451a-be2d-c2ebc9c13d43 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:42.116491 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c976bb9-11eb-451a-be2d-c2ebc9c13d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:42.116766 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c976bb9-11eb-451a-be2d-c2ebc9c13d43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:42.117454 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c976bb9-11eb-451a-be2d-c2ebc9c13d43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:42.119398 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c976bb9-11eb-451a-be2d-c2ebc9c13d43 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:42.121293 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1108/2207] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:42 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:42.125232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41912ca8-9750-4d16-9c3f-cddbec963271 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:42.144143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:42.153613 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1100/2208] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:42 2022] POST /identity/v3/projects => generated 393 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:05:42.156224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c976bb9-11eb-451a-be2d-c2ebc9c13d43 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:42.201423 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41912ca8-9750-4d16-9c3f-cddbec963271 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:42.222436 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c976bb9-11eb-451a-be2d-c2ebc9c13d43 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:42.228381 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d32bd-7b6d-4845-9c94-4e26dd637ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:42.229051 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d32bd-7b6d-4845-9c94-4e26dd637ee4 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:42.229332 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d32bd-7b6d-4845-9c94-4e26dd637ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:42.229637 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d32bd-7b6d-4845-9c94-4e26dd637ee4 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:42.230332 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6d32bd-7b6d-4845-9c94-4e26dd637ee4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:42.232254 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6d32bd-7b6d-4845-9c94-4e26dd637ee4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:42.248153 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:42.254668 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1101/2209] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:42 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:05:42.258650 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d32bd-7b6d-4845-9c94-4e26dd637ee4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:42.268138 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8544ffb-ceee-4342-862c-1a77b75c8a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:42.269001 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8544ffb-ceee-4342-862c-1a77b75c8a28 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:42.269361 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8544ffb-ceee-4342-862c-1a77b75c8a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:42.269739 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8544ffb-ceee-4342-862c-1a77b75c8a28 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:42.269858 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d32bd-7b6d-4845-9c94-4e26dd637ee4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:42.271023 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8544ffb-ceee-4342-862c-1a77b75c8a28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:42.273277 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8544ffb-ceee-4342-862c-1a77b75c8a28 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:42.275098 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca193837-b4ac-4160-ae20-527f65244ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:42.275754 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca193837-b4ac-4160-ae20-527f65244ff9 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:42.276042 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca193837-b4ac-4160-ae20-527f65244ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:42.276321 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca193837-b4ac-4160-ae20-527f65244ff9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:42.277044 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca193837-b4ac-4160-ae20-527f65244ff9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:42.278661 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca193837-b4ac-4160-ae20-527f65244ff9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:42.293270 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1102/2210] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 16:05:42.297470 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca193837-b4ac-4160-ae20-527f65244ff9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:42.298997 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1109/2211] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:42 2022] POST /identity/v3/users => generated 288 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:05:42.302776 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8544ffb-ceee-4342-862c-1a77b75c8a28 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:42.310494 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca193837-b4ac-4160-ae20-527f65244ff9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:42.313779 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8544ffb-ceee-4342-862c-1a77b75c8a28 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:42.315349 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec5bf0-ff5e-4a8d-b012-23b71173926e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:42.316019 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec5bf0-ff5e-4a8d-b012-23b71173926e admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:42.316307 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec5bf0-ff5e-4a8d-b012-23b71173926e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:42.316591 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec5bf0-ff5e-4a8d-b012-23b71173926e admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:42.317328 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec5bf0-ff5e-4a8d-b012-23b71173926e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:42.318446 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8686c335-3e6b-418e-beb7-7d19e1f071de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:42.318994 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec5bf0-ff5e-4a8d-b012-23b71173926e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:42.319105 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686c335-3e6b-418e-beb7-7d19e1f071de admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:42.319384 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686c335-3e6b-418e-beb7-7d19e1f071de admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:42.319641 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686c335-3e6b-418e-beb7-7d19e1f071de admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:42.320783 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8686c335-3e6b-418e-beb7-7d19e1f071de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:42.322414 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8686c335-3e6b-418e-beb7-7d19e1f071de admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:42.331215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1103/2212] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 16:05:42.335841 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1110/2213] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 16:05:42.336759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec5bf0-ff5e-4a8d-b012-23b71173926e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:42.339496 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8686c335-3e6b-418e-beb7-7d19e1f071de admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:42.346754 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec5bf0-ff5e-4a8d-b012-23b71173926e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:42.349299 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8686c335-3e6b-418e-beb7-7d19e1f071de admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:42.352329 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa279d6d-2a03-4daa-934e-c18b5bcefdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:42.353206 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa279d6d-2a03-4daa-934e-c18b5bcefdaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:42.353491 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa279d6d-2a03-4daa-934e-c18b5bcefdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:42.353797 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa279d6d-2a03-4daa-934e-c18b5bcefdaf admin admin] PATH_INFO: `/v3/projects/b31d4be9786d474fa957cee32e1475ad/users/57361619d5c34441b16ad617a4fe22d8/roles/54195e660fc349978b39d5794765a095` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:42.354652 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60379ad-65e2-4218-b9f2-647cdf142577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:42.355289 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60379ad-65e2-4218-b9f2-647cdf142577 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:42.355571 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60379ad-65e2-4218-b9f2-647cdf142577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:42.355845 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60379ad-65e2-4218-b9f2-647cdf142577 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:42.356564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60379ad-65e2-4218-b9f2-647cdf142577 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:42.358189 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60379ad-65e2-4218-b9f2-647cdf142577 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:42.368602 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1111/2214] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 16:05:42.368765 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa279d6d-2a03-4daa-934e-c18b5bcefdaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b31d4be9786d474fa957cee32e1475ad, user_id=57361619d5c34441b16ad617a4fe22d8, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:42.371514 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa279d6d-2a03-4daa-934e-c18b5bcefdaf admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:42.372777 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60379ad-65e2-4218-b9f2-647cdf142577 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:42.383339 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60379ad-65e2-4218-b9f2-647cdf142577 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:42.385048 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1104/2215] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:42 2022] PUT /identity/v3/projects/b31d4be9786d474fa957cee32e1475ad/users/57361619d5c34441b16ad617a4fe22d8/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:42.389906 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd0bdd-2376-4042-ab27-2fb8b5b05240 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:42.390205 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd0bdd-2376-4042-ab27-2fb8b5b05240 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:42.390505 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd0bdd-2376-4042-ab27-2fb8b5b05240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:42.407893 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f7bd0bdd-2376-4042-ab27-2fb8b5b05240 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:42.409396 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f7bd0bdd-2376-4042-ab27-2fb8b5b05240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:42.439841 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da73226e-2b47-48e3-8aec-6d696dc3e0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:42.440757 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da73226e-2b47-48e3-8aec-6d696dc3e0f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:42.441041 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da73226e-2b47-48e3-8aec-6d696dc3e0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:42.441357 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da73226e-2b47-48e3-8aec-6d696dc3e0f8 admin admin] PATH_INFO: `/v3/projects/7acfd0b52e544c59a62e7d15e94d99ec/users/280c7a2aae73450d82bf455810582627/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:42.455927 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da73226e-2b47-48e3-8aec-6d696dc3e0f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7acfd0b52e544c59a62e7d15e94d99ec, user_id=280c7a2aae73450d82bf455810582627, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:42.458639 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da73226e-2b47-48e3-8aec-6d696dc3e0f8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:42.469684 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1105/2216] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:42.472505 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1112/2217] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:42 2022] PUT /identity/v3/projects/7acfd0b52e544c59a62e7d15e94d99ec/users/280c7a2aae73450d82bf455810582627/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:05:42.480769 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd0bdd-2376-4042-ab27-2fb8b5b05240 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:42.482311 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f7bd0bdd-2376-4042-ab27-2fb8b5b05240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:42.484077 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142fc4a-9744-4e25-9dd5-9ae6190d8cc5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:42.484440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142fc4a-9744-4e25-9dd5-9ae6190d8cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:42.484731 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142fc4a-9744-4e25-9dd5-9ae6190d8cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:42.511065 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5142fc4a-9744-4e25-9dd5-9ae6190d8cc5 None None] MFA Rules not processed for user `280c7a2aae73450d82bf455810582627`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:42.512288 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5142fc4a-9744-4e25-9dd5-9ae6190d8cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:42.528378 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:42.541136 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd0bdd-2376-4042-ab27-2fb8b5b05240 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:42.578771 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1113/2218] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:05:42.583351 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5142fc4a-9744-4e25-9dd5-9ae6190d8cc5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:42.588307 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda9f89d-1a9b-4c6e-925a-c6715d827472 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:42.588959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda9f89d-1a9b-4c6e-925a-c6715d827472 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:42.589248 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda9f89d-1a9b-4c6e-925a-c6715d827472 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:42.589548 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda9f89d-1a9b-4c6e-925a-c6715d827472 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:42.603339 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:42.648543 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5142fc4a-9744-4e25-9dd5-9ae6190d8cc5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:42.651733 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda9f89d-1a9b-4c6e-925a-c6715d827472 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:42.653217 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda9f89d-1a9b-4c6e-925a-c6715d827472 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:42.653547 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca7f3b-347a-43b7-b6e1-ce71ea03f388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:42.654345 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca7f3b-347a-43b7-b6e1-ce71ea03f388 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:42.654632 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca7f3b-347a-43b7-b6e1-ce71ea03f388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:42.654909 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca7f3b-347a-43b7-b6e1-ce71ea03f388 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:42.655605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca7f3b-347a-43b7-b6e1-ce71ea03f388 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:42.657752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca7f3b-347a-43b7-b6e1-ce71ea03f388 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:42.668877 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1106/2219] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:42.687906 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1114/2220] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:42 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:05:42.691991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda9f89d-1a9b-4c6e-925a-c6715d827472 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:42.702149 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda9f89d-1a9b-4c6e-925a-c6715d827472 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:42.766153 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5416ee-1ae1-437a-a2f4-3642acf2f4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:42.766836 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5416ee-1ae1-437a-a2f4-3642acf2f4c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:42.767124 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5416ee-1ae1-437a-a2f4-3642acf2f4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:42.767395 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5416ee-1ae1-437a-a2f4-3642acf2f4c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:42.768105 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5416ee-1ae1-437a-a2f4-3642acf2f4c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:42.770118 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5416ee-1ae1-437a-a2f4-3642acf2f4c7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:42.781715 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca7f3b-347a-43b7-b6e1-ce71ea03f388 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:42.784451 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b4ca7f3b-347a-43b7-b6e1-ce71ea03f388 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:42.793636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1107/2221] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:42 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:42.797002 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5416ee-1ae1-437a-a2f4-3642acf2f4c7 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:42.798394 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca7f3b-347a-43b7-b6e1-ce71ea03f388 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:42.803514 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae5476-390f-4877-8682-c1ddd7223b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:42.804481 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae5476-390f-4877-8682-c1ddd7223b92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:42.805068 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae5476-390f-4877-8682-c1ddd7223b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:42.805473 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae5476-390f-4877-8682-c1ddd7223b92 admin admin] PATH_INFO: `/v3/users/147f52c566104bffb04e9d12e4179c13` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:42.806998 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ae5476-390f-4877-8682-c1ddd7223b92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=147f52c566104bffb04e9d12e4179c13)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:42.809162 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ae5476-390f-4877-8682-c1ddd7223b92 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:42.812287 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5416ee-1ae1-437a-a2f4-3642acf2f4c7 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:42.820439 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca29b272-8ebd-4944-a464-a91f5dbdaa62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:42.821399 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca29b272-8ebd-4944-a464-a91f5dbdaa62 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:42.821701 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca29b272-8ebd-4944-a464-a91f5dbdaa62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:42.822086 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca29b272-8ebd-4944-a464-a91f5dbdaa62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:42.822756 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca29b272-8ebd-4944-a464-a91f5dbdaa62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:42.824320 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca29b272-8ebd-4944-a464-a91f5dbdaa62 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:42.838356 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1108/2222] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:42.842014 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca29b272-8ebd-4944-a464-a91f5dbdaa62 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:42.856158 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca29b272-8ebd-4944-a464-a91f5dbdaa62 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:42.869191 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2316c2-946a-481d-bf6b-51d7de9f34ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:42.869191 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2316c2-946a-481d-bf6b-51d7de9f34ab admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:42.869191 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2316c2-946a-481d-bf6b-51d7de9f34ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:42.869191 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2316c2-946a-481d-bf6b-51d7de9f34ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:42.869608 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2316c2-946a-481d-bf6b-51d7de9f34ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:42.869608 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2316c2-946a-481d-bf6b-51d7de9f34ab admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:42.878426 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1109/2223] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 16:05:42.880203 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-90ae5476-390f-4877-8682-c1ddd7223b92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '147f52c566104bffb04e9d12e4179c13'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:42.881530 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-90ae5476-390f-4877-8682-c1ddd7223b92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '147f52c566104bffb04e9d12e4179c13'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:42.882267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-90ae5476-390f-4877-8682-c1ddd7223b92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '147f52c566104bffb04e9d12e4179c13'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:42.882999 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2316c2-946a-481d-bf6b-51d7de9f34ab admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:42.884849 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3d2316c2-946a-481d-bf6b-51d7de9f34ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:42.891710 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-90ae5476-390f-4877-8682-c1ddd7223b92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '147f52c566104bffb04e9d12e4179c13'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:42.898702 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2316c2-946a-481d-bf6b-51d7de9f34ab admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:42.901982 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-90ae5476-390f-4877-8682-c1ddd7223b92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '147f52c566104bffb04e9d12e4179c13'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:42.904242 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591abd61-5fa2-4b14-aa19-1e81ec5f4212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:42.905478 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591abd61-5fa2-4b14-aa19-1e81ec5f4212 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:42.905478 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591abd61-5fa2-4b14-aa19-1e81ec5f4212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:42.905780 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591abd61-5fa2-4b14-aa19-1e81ec5f4212 admin admin] PATH_INFO: `/v3/projects/434be561fe904fdc8a3eb1d7562d404b/users/ae651794b182425d8ebc68e6740bcfc0/roles/54195e660fc349978b39d5794765a095` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:42.922793 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591abd61-5fa2-4b14-aa19-1e81ec5f4212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=434be561fe904fdc8a3eb1d7562d404b, user_id=ae651794b182425d8ebc68e6740bcfc0, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:42.925450 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591abd61-5fa2-4b14-aa19-1e81ec5f4212 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:42.928025 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:42.929254 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1115/2224] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:42 2022] DELETE /identity/v3/users/147f52c566104bffb04e9d12e4179c13 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:05:42.931238 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e604a-4c99-4dba-985f-1b28b738db07 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:42.931502 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e604a-4c99-4dba-985f-1b28b738db07 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:42.931791 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e604a-4c99-4dba-985f-1b28b738db07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:42.964056 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1110/2225] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:42 2022] PUT /identity/v3/projects/434be561fe904fdc8a3eb1d7562d404b/users/ae651794b182425d8ebc68e6740bcfc0/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:05:42.965543 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc8764-b18c-4e07-91ae-e6f64fcd6cc8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:42.965862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc8764-b18c-4e07-91ae-e6f64fcd6cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:42.966177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc8764-b18c-4e07-91ae-e6f64fcd6cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:42.966855 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-188e604a-4c99-4dba-985f-1b28b738db07 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:42.968220 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-188e604a-4c99-4dba-985f-1b28b738db07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:42.996209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e8fc8764-b18c-4e07-91ae-e6f64fcd6cc8 None None] MFA Rules not processed for user `ae651794b182425d8ebc68e6740bcfc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:42.997771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e8fc8764-b18c-4e07-91ae-e6f64fcd6cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:43.046524 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1116/2226] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:43.071597 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1111/2227] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:43.076217 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e604a-4c99-4dba-985f-1b28b738db07 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:43.078390 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc8764-b18c-4e07-91ae-e6f64fcd6cc8 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:43.080082 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e8fc8764-b18c-4e07-91ae-e6f64fcd6cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:43.118962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e604a-4c99-4dba-985f-1b28b738db07 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:43.140739 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c0a38f-9d7f-4858-989f-51187976b9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:43.141704 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0a38f-9d7f-4858-989f-51187976b9ae admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:43.142618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0a38f-9d7f-4858-989f-51187976b9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:43.143090 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0a38f-9d7f-4858-989f-51187976b9ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:43.144012 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c0a38f-9d7f-4858-989f-51187976b9ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:43.146849 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c0a38f-9d7f-4858-989f-51187976b9ae admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:43.162242 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:43.174589 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc8764-b18c-4e07-91ae-e6f64fcd6cc8 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:43.181799 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b87484b-09c1-48c5-8c41-52b3f4ca903d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:43.182478 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87484b-09c1-48c5-8c41-52b3f4ca903d service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:43.182771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87484b-09c1-48c5-8c41-52b3f4ca903d service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:43.183417 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87484b-09c1-48c5-8c41-52b3f4ca903d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:43.186255 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1117/2228] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:43 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:43.188587 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c0a38f-9d7f-4858-989f-51187976b9ae admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:43.190815 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a6c0a38f-9d7f-4858-989f-51187976b9ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:43.196772 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b87484b-09c1-48c5-8c41-52b3f4ca903d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:43.198464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b87484b-09c1-48c5-8c41-52b3f4ca903d service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:43.209413 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1112/2229] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 16:05:43.210242 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b87484b-09c1-48c5-8c41-52b3f4ca903d service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:43.224688 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b87484b-09c1-48c5-8c41-52b3f4ca903d service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:43.295684 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c0a38f-9d7f-4858-989f-51187976b9ae admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:43.301527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6144661a-b99f-4cbc-b491-2e3a772d9853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:43.302564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6144661a-b99f-4cbc-b491-2e3a772d9853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:43.303377 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6144661a-b99f-4cbc-b491-2e3a772d9853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:43.303760 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6144661a-b99f-4cbc-b491-2e3a772d9853 admin admin] PATH_INFO: `/v3/users/3dddd8bba6bc4bd588a772b2d5969ce1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:43.307393 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6144661a-b99f-4cbc-b491-2e3a772d9853 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dddd8bba6bc4bd588a772b2d5969ce1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:43.310785 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6144661a-b99f-4cbc-b491-2e3a772d9853 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:43.340463 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c06a9-cb1f-4a9f-b96c-5b1b5c6a23af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:43.342050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c06a9-cb1f-4a9f-b96c-5b1b5c6a23af admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:43.343203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c06a9-cb1f-4a9f-b96c-5b1b5c6a23af admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:43.343423 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c06a9-cb1f-4a9f-b96c-5b1b5c6a23af admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:43.344262 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33c06a9-cb1f-4a9f-b96c-5b1b5c6a23af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:43.346548 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33c06a9-cb1f-4a9f-b96c-5b1b5c6a23af admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:43.373457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1113/2230] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:43 2022] POST /identity/v3/users => generated 295 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:43.377845 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c06a9-cb1f-4a9f-b96c-5b1b5c6a23af admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:43.379559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6144661a-b99f-4cbc-b491-2e3a772d9853 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dddd8bba6bc4bd588a772b2d5969ce1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:43.380512 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6144661a-b99f-4cbc-b491-2e3a772d9853 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dddd8bba6bc4bd588a772b2d5969ce1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:43.381971 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6144661a-b99f-4cbc-b491-2e3a772d9853 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dddd8bba6bc4bd588a772b2d5969ce1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:43.392334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6144661a-b99f-4cbc-b491-2e3a772d9853 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dddd8bba6bc4bd588a772b2d5969ce1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:43.394143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c06a9-cb1f-4a9f-b96c-5b1b5c6a23af admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:43.404761 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa68549-088a-4b4b-8ea8-21ede78a46fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:43.405495 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa68549-088a-4b4b-8ea8-21ede78a46fe admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:43.405814 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa68549-088a-4b4b-8ea8-21ede78a46fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:43.406092 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa68549-088a-4b4b-8ea8-21ede78a46fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:43.407506 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa68549-088a-4b4b-8ea8-21ede78a46fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:43.410069 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa68549-088a-4b4b-8ea8-21ede78a46fe admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:43.419800 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6144661a-b99f-4cbc-b491-2e3a772d9853 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dddd8bba6bc4bd588a772b2d5969ce1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:43.429244 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1114/2231] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:43.430141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa68549-088a-4b4b-8ea8-21ede78a46fe admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:43.432981 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cfa68549-088a-4b4b-8ea8-21ede78a46fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:43.445083 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:43.449272 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa68549-088a-4b4b-8ea8-21ede78a46fe admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:43.454212 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1118/2232] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:43 2022] DELETE /identity/v3/users/3dddd8bba6bc4bd588a772b2d5969ce1 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:43.455787 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1bb87-8318-4646-8a77-62924883c185 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:43.456186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1bb87-8318-4646-8a77-62924883c185 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:43.456566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1bb87-8318-4646-8a77-62924883c185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:43.493946 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2fb1bb87-8318-4646-8a77-62924883c185 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:43.496717 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2fb1bb87-8318-4646-8a77-62924883c185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:43.531821 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a639e-6700-4f5b-99d2-59c89051eace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:43.533037 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a639e-6700-4f5b-99d2-59c89051eace admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:43.533423 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a639e-6700-4f5b-99d2-59c89051eace admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:43.533952 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a639e-6700-4f5b-99d2-59c89051eace admin admin] PATH_INFO: `/v3/projects/b46d01a124c94769bda9d27bab07b657/users/f14b1aa7cc7245f59312350c29adcea8/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:43.552411 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672a639e-6700-4f5b-99d2-59c89051eace admin admin] RBAC: Authorizing `identity:create_grant(project_id=b46d01a124c94769bda9d27bab07b657, user_id=f14b1aa7cc7245f59312350c29adcea8, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:43.556186 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672a639e-6700-4f5b-99d2-59c89051eace admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:43.574975 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1115/2233] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:43 2022] PUT /identity/v3/projects/b46d01a124c94769bda9d27bab07b657/users/f14b1aa7cc7245f59312350c29adcea8/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:43.578900 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a639e-6700-4f5b-99d2-59c89051eace admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:43.587267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1119/2234] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:43.617215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1bb87-8318-4646-8a77-62924883c185 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:43.622960 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2fb1bb87-8318-4646-8a77-62924883c185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:43.656941 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:43.676215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a639e-6700-4f5b-99d2-59c89051eace admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:43.683624 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d6d6c-ddcf-4a95-9985-029a9faeec45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:43.684476 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d6d6c-ddcf-4a95-9985-029a9faeec45 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:43.685373 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d6d6c-ddcf-4a95-9985-029a9faeec45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:43.692094 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d6d6c-ddcf-4a95-9985-029a9faeec45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:43.693027 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98d6d6c-ddcf-4a95-9985-029a9faeec45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:43.697145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98d6d6c-ddcf-4a95-9985-029a9faeec45 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:43.715762 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1116/2235] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 16:05:43.721092 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d6d6c-ddcf-4a95-9985-029a9faeec45 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:43.751846 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d6d6c-ddcf-4a95-9985-029a9faeec45 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:43.766788 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3367c1-058f-466e-b27f-f598e90ef8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:43.769657 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3367c1-058f-466e-b27f-f598e90ef8a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:43.770656 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3367c1-058f-466e-b27f-f598e90ef8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:43.770979 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3367c1-058f-466e-b27f-f598e90ef8a8 admin admin] PATH_INFO: `/v3/projects/b46d01a124c94769bda9d27bab07b657/users/f14b1aa7cc7245f59312350c29adcea8/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:43.777027 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3367c1-058f-466e-b27f-f598e90ef8a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b46d01a124c94769bda9d27bab07b657, user_id=f14b1aa7cc7245f59312350c29adcea8, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:43.783292 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3367c1-058f-466e-b27f-f598e90ef8a8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:43.785133 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1bb87-8318-4646-8a77-62924883c185 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:43.810350 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1117/2236] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:43 2022] PUT /identity/v3/projects/b46d01a124c94769bda9d27bab07b657/users/f14b1aa7cc7245f59312350c29adcea8/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:05:43.823405 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcfd0f-0c64-4992-8135-80766c730de4 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:43.828664 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcfd0f-0c64-4992-8135-80766c730de4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:43.829001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcfd0f-0c64-4992-8135-80766c730de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:43.891542 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516d42ee-b425-4c96-a819-77b95b9cdb2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:43.892470 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516d42ee-b425-4c96-a819-77b95b9cdb2b service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:43.893012 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516d42ee-b425-4c96-a819-77b95b9cdb2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:43.893341 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516d42ee-b425-4c96-a819-77b95b9cdb2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:43.909386 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c8fcfd0f-0c64-4992-8135-80766c730de4 None None] MFA Rules not processed for user `f14b1aa7cc7245f59312350c29adcea8`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:43.910924 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c8fcfd0f-0c64-4992-8135-80766c730de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:43.932414 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516d42ee-b425-4c96-a819-77b95b9cdb2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:43.937731 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516d42ee-b425-4c96-a819-77b95b9cdb2b service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:43.960174 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1120/2237] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:43.963017 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516d42ee-b425-4c96-a819-77b95b9cdb2b service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:43.967540 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-516d42ee-b425-4c96-a819-77b95b9cdb2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:43.996748 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516d42ee-b425-4c96-a819-77b95b9cdb2b service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:44.034931 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:44.049219 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1118/2238] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:43 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:44.061584 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fcfd0f-0c64-4992-8135-80766c730de4 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:44.063790 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c8fcfd0f-0c64-4992-8135-80766c730de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:44.104770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35368b5-6339-47ea-818e-cf71152a9813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:44.107791 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35368b5-6339-47ea-818e-cf71152a9813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:44.108478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35368b5-6339-47ea-818e-cf71152a9813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:44.109640 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35368b5-6339-47ea-818e-cf71152a9813 admin admin] PATH_INFO: `/v3/projects/b782b3c15e134b2bab26628b2286f96d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:44.113953 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35368b5-6339-47ea-818e-cf71152a9813 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b782b3c15e134b2bab26628b2286f96d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:44.119521 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35368b5-6339-47ea-818e-cf71152a9813 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:44.204928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fcfd0f-0c64-4992-8135-80766c730de4 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:44.222339 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a35368b5-6339-47ea-818e-cf71152a9813 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b782b3c15e134b2bab26628b2286f96d', 'request_id': 'req-a35368b5-6339-47ea-818e-cf71152a9813'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:44.236021 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a35368b5-6339-47ea-818e-cf71152a9813 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b782b3c15e134b2bab26628b2286f96d', 'request_id': 'req-a35368b5-6339-47ea-818e-cf71152a9813'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:44.247078 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a35368b5-6339-47ea-818e-cf71152a9813 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b782b3c15e134b2bab26628b2286f96d', 'request_id': 'req-a35368b5-6339-47ea-818e-cf71152a9813'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:44.268226 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a35368b5-6339-47ea-818e-cf71152a9813 admin admin] The token cache is being invalidate because project b782b3c15e134b2bab26628b2286f96d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:44.268662 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a35368b5-6339-47ea-818e-cf71152a9813 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:44.269479 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a35368b5-6339-47ea-818e-cf71152a9813 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:44.272672 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1121/2239] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:43 2022] DELETE /identity/v3/projects/b782b3c15e134b2bab26628b2286f96d => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:44.279241 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2aeebd-e15f-4021-b759-0bfa52ee73c7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:44.279646 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2aeebd-e15f-4021-b759-0bfa52ee73c7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:44.279997 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2aeebd-e15f-4021-b759-0bfa52ee73c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:44.284423 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf956fc-e12e-4682-9adf-f0d9c742c6e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:44.285169 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf956fc-e12e-4682-9adf-f0d9c742c6e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:44.285533 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf956fc-e12e-4682-9adf-f0d9c742c6e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:44.285951 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf956fc-e12e-4682-9adf-f0d9c742c6e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:44.288074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-caf956fc-e12e-4682-9adf-f0d9c742c6e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:44.293073 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:44.311792 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-bf2aeebd-e15f-4021-b759-0bfa52ee73c7 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:44.314133 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bf2aeebd-e15f-4021-b759-0bfa52ee73c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:44.399549 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf956fc-e12e-4682-9adf-f0d9c742c6e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:44.403672 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf956fc-e12e-4682-9adf-f0d9c742c6e3 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:44.428383 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1122/2240] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:44.433937 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37715e03-5062-4227-a95a-da683d23fb0d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:44.434290 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37715e03-5062-4227-a95a-da683d23fb0d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:44.434443 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1119/2241] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:44.434641 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37715e03-5062-4227-a95a-da683d23fb0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:44.468305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-37715e03-5062-4227-a95a-da683d23fb0d None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:44.471346 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-37715e03-5062-4227-a95a-da683d23fb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:44.488406 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1123/2242] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:44.489936 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf956fc-e12e-4682-9adf-f0d9c742c6e3 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:44.504152 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf956fc-e12e-4682-9adf-f0d9c742c6e3 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:44.512219 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aaa8a2-9704-4b2b-aeac-4644d12d0c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:44.513488 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aaa8a2-9704-4b2b-aeac-4644d12d0c45 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:44.513488 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aaa8a2-9704-4b2b-aeac-4644d12d0c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:44.513908 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aaa8a2-9704-4b2b-aeac-4644d12d0c45 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:44.514982 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aaa8a2-9704-4b2b-aeac-4644d12d0c45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:05:44.515732 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aaa8a2-9704-4b2b-aeac-4644d12d0c45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:44.517315 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aaa8a2-9704-4b2b-aeac-4644d12d0c45 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:44.537058 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1120/2243] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:05:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:05:44.542381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37715e03-5062-4227-a95a-da683d23fb0d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:44.554650 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37715e03-5062-4227-a95a-da683d23fb0d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:44.567125 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49aa6f2-49ec-4542-9c60-068f5a4846b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:44.570987 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49aa6f2-49ec-4542-9c60-068f5a4846b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:44.570987 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49aa6f2-49ec-4542-9c60-068f5a4846b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:44.570987 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49aa6f2-49ec-4542-9c60-068f5a4846b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:44.570987 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49aa6f2-49ec-4542-9c60-068f5a4846b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:44.571988 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aaa8a2-9704-4b2b-aeac-4644d12d0c45 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:44.573946 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e1aaa8a2-9704-4b2b-aeac-4644d12d0c45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:44.589754 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49aa6f2-49ec-4542-9c60-068f5a4846b9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:44.596616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aaa8a2-9704-4b2b-aeac-4644d12d0c45 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:44.618395 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a93619-fcbb-4392-85d6-6471d58796d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:44.619456 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a93619-fcbb-4392-85d6-6471d58796d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:44.619594 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a93619-fcbb-4392-85d6-6471d58796d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:44.619847 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a93619-fcbb-4392-85d6-6471d58796d8 admin admin] PATH_INFO: `/v3/projects/bb28d25de0df42c49f504c609f87efae` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:44.623529 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a93619-fcbb-4392-85d6-6471d58796d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb28d25de0df42c49f504c609f87efae)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:44.625595 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a93619-fcbb-4392-85d6-6471d58796d8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:44.648455 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1124/2244] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:44 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:44.649591 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49aa6f2-49ec-4542-9c60-068f5a4846b9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:44.774084 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:44.779678 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c1a93619-fcbb-4392-85d6-6471d58796d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb28d25de0df42c49f504c609f87efae', 'request_id': 'req-c1a93619-fcbb-4392-85d6-6471d58796d8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:44.794478 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c1a93619-fcbb-4392-85d6-6471d58796d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb28d25de0df42c49f504c609f87efae', 'request_id': 'req-c1a93619-fcbb-4392-85d6-6471d58796d8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:44.795165 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49aa6f2-49ec-4542-9c60-068f5a4846b9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:44.810547 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c1a93619-fcbb-4392-85d6-6471d58796d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb28d25de0df42c49f504c609f87efae', 'request_id': 'req-c1a93619-fcbb-4392-85d6-6471d58796d8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:44.845307 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c1a93619-fcbb-4392-85d6-6471d58796d8 admin admin] The token cache is being invalidate because project bb28d25de0df42c49f504c609f87efae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:44.846055 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c1a93619-fcbb-4392-85d6-6471d58796d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:44.846873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c1a93619-fcbb-4392-85d6-6471d58796d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:44.849053 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1121/2245] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:44 2022] DELETE /identity/v3/projects/bb28d25de0df42c49f504c609f87efae => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:44.853224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a93619-fcbb-4392-85d6-6471d58796d8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:44.854590 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c1a93619-fcbb-4392-85d6-6471d58796d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:44.915719 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a47a4-1413-4c18-83f8-b23f9dc43d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:44.916464 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a47a4-1413-4c18-83f8-b23f9dc43d1e admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:44.916779 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a47a4-1413-4c18-83f8-b23f9dc43d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:44.917096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a47a4-1413-4c18-83f8-b23f9dc43d1e admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:44.918547 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5a47a4-1413-4c18-83f8-b23f9dc43d1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:44.920516 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5a47a4-1413-4c18-83f8-b23f9dc43d1e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:44.973939 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1125/2246] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:44 2022] POST /identity/v3/users => generated 288 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:44.975568 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a47a4-1413-4c18-83f8-b23f9dc43d1e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:44.979593 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6e5a47a4-1413-4c18-83f8-b23f9dc43d1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:44.992445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a93619-fcbb-4392-85d6-6471d58796d8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:44.998755 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e74151-6f14-4012-884f-71088297c34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:44.999468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e74151-6f14-4012-884f-71088297c34f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:44.999767 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e74151-6f14-4012-884f-71088297c34f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:45.001151 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e74151-6f14-4012-884f-71088297c34f admin admin] PATH_INFO: `/v3/users/6962f8afe6f9496c95a345132ed9d23b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:45.002697 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e74151-6f14-4012-884f-71088297c34f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6962f8afe6f9496c95a345132ed9d23b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:45.004817 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e74151-6f14-4012-884f-71088297c34f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:45.011258 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a47a4-1413-4c18-83f8-b23f9dc43d1e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:45.021994 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef59d10-e5d8-4043-ab54-2507e8f4ef03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:45.022933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef59d10-e5d8-4043-ab54-2507e8f4ef03 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:45.023529 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef59d10-e5d8-4043-ab54-2507e8f4ef03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:45.023966 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef59d10-e5d8-4043-ab54-2507e8f4ef03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:45.024905 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef59d10-e5d8-4043-ab54-2507e8f4ef03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:45.026987 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef59d10-e5d8-4043-ab54-2507e8f4ef03 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:45.035752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:45.050226 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1126/2247] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:45.051849 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef59d10-e5d8-4043-ab54-2507e8f4ef03 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:45.063182 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef59d10-e5d8-4043-ab54-2507e8f4ef03 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:45.068114 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0042cf6-6875-4712-8b82-52ed7382f77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:45.068840 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0042cf6-6875-4712-8b82-52ed7382f77a admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:45.069132 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0042cf6-6875-4712-8b82-52ed7382f77a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:45.069424 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0042cf6-6875-4712-8b82-52ed7382f77a admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:45.070202 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0042cf6-6875-4712-8b82-52ed7382f77a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:45.071837 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0042cf6-6875-4712-8b82-52ed7382f77a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:45.076531 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d8e74151-6f14-4012-884f-71088297c34f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6962f8afe6f9496c95a345132ed9d23b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:45.078203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d8e74151-6f14-4012-884f-71088297c34f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6962f8afe6f9496c95a345132ed9d23b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:45.079290 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d8e74151-6f14-4012-884f-71088297c34f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6962f8afe6f9496c95a345132ed9d23b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:45.083701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1127/2248] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:45.087843 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0042cf6-6875-4712-8b82-52ed7382f77a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:45.090475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c0042cf6-6875-4712-8b82-52ed7382f77a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:45.092232 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d8e74151-6f14-4012-884f-71088297c34f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6962f8afe6f9496c95a345132ed9d23b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:45.106164 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d8e74151-6f14-4012-884f-71088297c34f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6962f8afe6f9496c95a345132ed9d23b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:45.108181 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0042cf6-6875-4712-8b82-52ed7382f77a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:45.113402 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2713222d-a14c-4a40-a59f-3573b619067d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:45.114346 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713222d-a14c-4a40-a59f-3573b619067d admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:45.114634 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713222d-a14c-4a40-a59f-3573b619067d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:45.114962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713222d-a14c-4a40-a59f-3573b619067d admin admin] PATH_INFO: `/v3/projects/4d1c1ee916984d4f86e3c7c3543b8154/users/6a966c72cd384d379802a23407abdebb/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:45.131580 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2713222d-a14c-4a40-a59f-3573b619067d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d1c1ee916984d4f86e3c7c3543b8154, user_id=6a966c72cd384d379802a23407abdebb, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:45.132496 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1122/2249] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:44 2022] DELETE /identity/v3/users/6962f8afe6f9496c95a345132ed9d23b => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:45.134651 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2713222d-a14c-4a40-a59f-3573b619067d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:45.155600 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1128/2250] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:45 2022] PUT /identity/v3/projects/4d1c1ee916984d4f86e3c7c3543b8154/users/6a966c72cd384d379802a23407abdebb/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:05:45.166590 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f13c7-f243-4c24-be0d-0669d468a157 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:45.166786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f13c7-f243-4c24-be0d-0669d468a157 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:45.167046 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f13c7-f243-4c24-be0d-0669d468a157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:45.249024 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5a0f13c7-f243-4c24-be0d-0669d468a157 None None] MFA Rules not processed for user `6a966c72cd384d379802a23407abdebb`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:45.251726 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5a0f13c7-f243-4c24-be0d-0669d468a157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:45.293463 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3c037-f068-42ea-8ba0-00bc33a91cb7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:45.293903 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3c037-f068-42ea-8ba0-00bc33a91cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:45.294272 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3c037-f068-42ea-8ba0-00bc33a91cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:45.333200 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e4c3c037-f068-42ea-8ba0-00bc33a91cb7 None None] MFA Rules not processed for user `ad718e6456d542d7a2d99c549909b4c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:45.335385 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e4c3c037-f068-42ea-8ba0-00bc33a91cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:45.417363 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:45.463169 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1129/2251] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:05:45.468969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1123/2252] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:45.481327 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3c037-f068-42ea-8ba0-00bc33a91cb7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:45.482848 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e4c3c037-f068-42ea-8ba0-00bc33a91cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:45.619809 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3c037-f068-42ea-8ba0-00bc33a91cb7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:45.629054 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f560477c-eb3b-4de3-ad7f-053f0a2211da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:45.630422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560477c-eb3b-4de3-ad7f-053f0a2211da service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:45.631411 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560477c-eb3b-4de3-ad7f-053f0a2211da service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:45.631886 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560477c-eb3b-4de3-ad7f-053f0a2211da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:45.644739 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f560477c-eb3b-4de3-ad7f-053f0a2211da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:45.647989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f560477c-eb3b-4de3-ad7f-053f0a2211da service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:45.660072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1130/2253] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:45.666147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f13c7-f243-4c24-be0d-0669d468a157 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:45.667642 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5a0f13c7-f243-4c24-be0d-0669d468a157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:45.813256 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f13c7-f243-4c24-be0d-0669d468a157 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:45.819211 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b8b5c6-3a0c-4fff-b288-d4a565049db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:45.820310 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8b5c6-3a0c-4fff-b288-d4a565049db5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:45.821241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8b5c6-3a0c-4fff-b288-d4a565049db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:45.821957 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8b5c6-3a0c-4fff-b288-d4a565049db5 admin admin] PATH_INFO: `/v3/projects/5ab753a4620346028af6fd5701c1da27` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:45.824908 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b8b5c6-3a0c-4fff-b288-d4a565049db5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ab753a4620346028af6fd5701c1da27)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:45.826540 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b8b5c6-3a0c-4fff-b288-d4a565049db5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:45.888104 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c4b8b5c6-3a0c-4fff-b288-d4a565049db5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ab753a4620346028af6fd5701c1da27', 'request_id': 'req-c4b8b5c6-3a0c-4fff-b288-d4a565049db5'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:45.914590 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c4b8b5c6-3a0c-4fff-b288-d4a565049db5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ab753a4620346028af6fd5701c1da27', 'request_id': 'req-c4b8b5c6-3a0c-4fff-b288-d4a565049db5'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:45.928256 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c4b8b5c6-3a0c-4fff-b288-d4a565049db5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ab753a4620346028af6fd5701c1da27', 'request_id': 'req-c4b8b5c6-3a0c-4fff-b288-d4a565049db5'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:45.952809 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c4b8b5c6-3a0c-4fff-b288-d4a565049db5 admin admin] The token cache is being invalidate because project 5ab753a4620346028af6fd5701c1da27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:45.953114 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c4b8b5c6-3a0c-4fff-b288-d4a565049db5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:45.953893 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c4b8b5c6-3a0c-4fff-b288-d4a565049db5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:45.957829 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1124/2254] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:45 2022] DELETE /identity/v3/projects/5ab753a4620346028af6fd5701c1da27 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:45.962110 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17a4b8-00cb-4c9c-84f3-2aec362c8b78 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:45.962628 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17a4b8-00cb-4c9c-84f3-2aec362c8b78 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:45.962924 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17a4b8-00cb-4c9c-84f3-2aec362c8b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:45.997643 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4c17a4b8-00cb-4c9c-84f3-2aec362c8b78 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:45.999830 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4c17a4b8-00cb-4c9c-84f3-2aec362c8b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:46.116139 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1131/2255] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:46.124040 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838ca65-a20a-4da1-9da0-5ec87b6dd42a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:46.124594 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838ca65-a20a-4da1-9da0-5ec87b6dd42a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:46.124850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838ca65-a20a-4da1-9da0-5ec87b6dd42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:46.180866 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7838ca65-a20a-4da1-9da0-5ec87b6dd42a None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:46.182507 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7838ca65-a20a-4da1-9da0-5ec87b6dd42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:46.200850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1125/2256] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:46.203849 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17a4b8-00cb-4c9c-84f3-2aec362c8b78 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:46.220905 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17a4b8-00cb-4c9c-84f3-2aec362c8b78 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:46.231553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e65041-f36f-498e-b4bf-9a808c63dfaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:46.233234 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e65041-f36f-498e-b4bf-9a808c63dfaa admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:46.233773 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e65041-f36f-498e-b4bf-9a808c63dfaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:46.234705 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e65041-f36f-498e-b4bf-9a808c63dfaa admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:46.238021 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e65041-f36f-498e-b4bf-9a808c63dfaa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:05:46.238946 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e65041-f36f-498e-b4bf-9a808c63dfaa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:46.246831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e65041-f36f-498e-b4bf-9a808c63dfaa admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:46.267851 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:46.277373 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1132/2257] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:05:46.282010 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838ca65-a20a-4da1-9da0-5ec87b6dd42a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:46.297985 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838ca65-a20a-4da1-9da0-5ec87b6dd42a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:46.318475 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95be797-0eb3-4deb-b222-cba5dab881d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:46.319179 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95be797-0eb3-4deb-b222-cba5dab881d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:46.319469 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95be797-0eb3-4deb-b222-cba5dab881d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:46.319753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95be797-0eb3-4deb-b222-cba5dab881d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:46.320476 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95be797-0eb3-4deb-b222-cba5dab881d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:46.328202 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95be797-0eb3-4deb-b222-cba5dab881d5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:46.335789 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e65041-f36f-498e-b4bf-9a808c63dfaa admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:46.337561 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-91e65041-f36f-498e-b4bf-9a808c63dfaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:46.374584 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1126/2258] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:46 2022] POST /identity/v3/projects => generated 407 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:46.378896 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95be797-0eb3-4deb-b222-cba5dab881d5 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:46.435447 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e65041-f36f-498e-b4bf-9a808c63dfaa admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:46.449910 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:46.470956 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95be797-0eb3-4deb-b222-cba5dab881d5 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:46.476874 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023ea8a-ea60-42c7-a610-bab9ce84ad8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:46.477810 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023ea8a-ea60-42c7-a610-bab9ce84ad8b admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:46.478252 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023ea8a-ea60-42c7-a610-bab9ce84ad8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:46.478680 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023ea8a-ea60-42c7-a610-bab9ce84ad8b admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:46.479523 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1023ea8a-ea60-42c7-a610-bab9ce84ad8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:46.481977 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1023ea8a-ea60-42c7-a610-bab9ce84ad8b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:46.503894 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f4382-4f1f-4e41-80db-91e0d7873204 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:46.505331 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f4382-4f1f-4e41-80db-91e0d7873204 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:46.506351 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f4382-4f1f-4e41-80db-91e0d7873204 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:46.507008 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f4382-4f1f-4e41-80db-91e0d7873204 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:46.509671 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-225f4382-4f1f-4e41-80db-91e0d7873204 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:46.527460 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1127/2259] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:46 2022] POST /identity/v3/users => generated 304 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:46.531998 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023ea8a-ea60-42c7-a610-bab9ce84ad8b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:46.545136 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023ea8a-ea60-42c7-a610-bab9ce84ad8b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:46.551332 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82054268-8a20-4308-970d-ab1069a9ebdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:46.552107 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82054268-8a20-4308-970d-ab1069a9ebdb admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:46.552490 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82054268-8a20-4308-970d-ab1069a9ebdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:46.552865 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82054268-8a20-4308-970d-ab1069a9ebdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:46.553960 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82054268-8a20-4308-970d-ab1069a9ebdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:46.556349 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82054268-8a20-4308-970d-ab1069a9ebdb admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:46.572100 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1128/2260] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:46.573415 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82054268-8a20-4308-970d-ab1069a9ebdb admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:46.586853 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82054268-8a20-4308-970d-ab1069a9ebdb admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:46.592454 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f0ff5-3ca8-4c16-b102-d4768d09fb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:46.593276 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f0ff5-3ca8-4c16-b102-d4768d09fb0a admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:46.593692 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f0ff5-3ca8-4c16-b102-d4768d09fb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:46.594406 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f0ff5-3ca8-4c16-b102-d4768d09fb0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:46.595249 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532f0ff5-3ca8-4c16-b102-d4768d09fb0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:46.597460 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532f0ff5-3ca8-4c16-b102-d4768d09fb0a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:46.599045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f4382-4f1f-4e41-80db-91e0d7873204 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:46.601718 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f4382-4f1f-4e41-80db-91e0d7873204 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:46.613785 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1129/2261] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:46.615828 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f0ff5-3ca8-4c16-b102-d4768d09fb0a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:46.617111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1133/2262] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 16:05:46.628424 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f0ff5-3ca8-4c16-b102-d4768d09fb0a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:46.635339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30bacd7-98f2-487f-8874-2df71b2a5511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:46.636395 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30bacd7-98f2-487f-8874-2df71b2a5511 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:46.636730 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30bacd7-98f2-487f-8874-2df71b2a5511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:46.637011 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30bacd7-98f2-487f-8874-2df71b2a5511 admin admin] PATH_INFO: `/v3/projects/3ca593f4cc714fa6a49cc55d8d3e91ad/users/213ff2b8cf36404fb4cf86d13463f6e8/roles/54195e660fc349978b39d5794765a095` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:46.658029 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30bacd7-98f2-487f-8874-2df71b2a5511 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ca593f4cc714fa6a49cc55d8d3e91ad, user_id=213ff2b8cf36404fb4cf86d13463f6e8, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:46.660989 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30bacd7-98f2-487f-8874-2df71b2a5511 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:46.686891 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1130/2263] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:46 2022] PUT /identity/v3/projects/3ca593f4cc714fa6a49cc55d8d3e91ad/users/213ff2b8cf36404fb4cf86d13463f6e8/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:46.688699 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365740a-47c5-4e06-84fa-81b7a0fe8567 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:46.689001 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365740a-47c5-4e06-84fa-81b7a0fe8567 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:46.689301 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365740a-47c5-4e06-84fa-81b7a0fe8567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:46.735073 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7365740a-47c5-4e06-84fa-81b7a0fe8567 None None] MFA Rules not processed for user `213ff2b8cf36404fb4cf86d13463f6e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:46.736333 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7365740a-47c5-4e06-84fa-81b7a0fe8567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:46.884115 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:46.908798 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30bacd7-98f2-487f-8874-2df71b2a5511 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:46.920774 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1134/2264] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:46 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:47.007354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30bacd7-98f2-487f-8874-2df71b2a5511 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:47.020850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dbecca-9642-41f4-9c86-e1737f6749b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:47.022371 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbecca-9642-41f4-9c86-e1737f6749b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:47.022703 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbecca-9642-41f4-9c86-e1737f6749b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:47.026384 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbecca-9642-41f4-9c86-e1737f6749b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:47.027249 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dbecca-9642-41f4-9c86-e1737f6749b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:47.030928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dbecca-9642-41f4-9c86-e1737f6749b6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:47.087914 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1131/2265] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:46 2022] POST /identity/v3/projects => generated 409 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:05:47.090182 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7365740a-47c5-4e06-84fa-81b7a0fe8567 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:47.108262 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7365740a-47c5-4e06-84fa-81b7a0fe8567 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:47.268902 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e02fa59-aec7-45bd-8a63-963009cbc563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:47.269629 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02fa59-aec7-45bd-8a63-963009cbc563 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:47.269930 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02fa59-aec7-45bd-8a63-963009cbc563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:47.270314 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02fa59-aec7-45bd-8a63-963009cbc563 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:47.271043 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e02fa59-aec7-45bd-8a63-963009cbc563 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:47.273140 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e02fa59-aec7-45bd-8a63-963009cbc563 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:47.350234 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dbecca-9642-41f4-9c86-e1737f6749b6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:47.360380 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1135/2266] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:47 2022] POST /identity/v3/users => generated 311 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:47.370185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dbecca-9642-41f4-9c86-e1737f6749b6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:47.385659 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8af74-55a6-4246-865d-12697e7ccf3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:47.386650 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8af74-55a6-4246-865d-12697e7ccf3a admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:47.387396 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8af74-55a6-4246-865d-12697e7ccf3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:47.390363 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8af74-55a6-4246-865d-12697e7ccf3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:47.392944 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8af74-55a6-4246-865d-12697e7ccf3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:47.396235 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8af74-55a6-4246-865d-12697e7ccf3a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:47.440884 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1132/2267] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:47 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:47.444713 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e02fa59-aec7-45bd-8a63-963009cbc563 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:47.458846 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e02fa59-aec7-45bd-8a63-963009cbc563 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:47.464887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7347b3-b3bb-490b-84c4-cf15069e27f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:47.466392 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7347b3-b3bb-490b-84c4-cf15069e27f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:47.466785 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7347b3-b3bb-490b-84c4-cf15069e27f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:47.467068 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7347b3-b3bb-490b-84c4-cf15069e27f2 admin admin] PATH_INFO: `/v3/projects/7bea7e03e85645e49794b7236aa8ad79/users/d76063c239164f8aa736b3dc792b9c60/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:47.493271 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7347b3-b3bb-490b-84c4-cf15069e27f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bea7e03e85645e49794b7236aa8ad79, user_id=d76063c239164f8aa736b3dc792b9c60, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:47.497597 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7347b3-b3bb-490b-84c4-cf15069e27f2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:47.519012 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1136/2268] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:47 2022] PUT /identity/v3/projects/7bea7e03e85645e49794b7236aa8ad79/users/d76063c239164f8aa736b3dc792b9c60/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:47.524050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8af74-55a6-4246-865d-12697e7ccf3a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:47.539150 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8af74-55a6-4246-865d-12697e7ccf3a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:47.681696 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb5859-ab32-43c9-be80-ab09723982f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:47.682842 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb5859-ab32-43c9-be80-ab09723982f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:47.683286 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb5859-ab32-43c9-be80-ab09723982f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:47.683717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb5859-ab32-43c9-be80-ab09723982f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:47.684749 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbb5859-ab32-43c9-be80-ab09723982f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:47.686644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbb5859-ab32-43c9-be80-ab09723982f0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:47.698972 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:47.718177 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7347b3-b3bb-490b-84c4-cf15069e27f2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:47.733659 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1133/2269] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:47 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:47.734673 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7347b3-b3bb-490b-84c4-cf15069e27f2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:47.746862 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4934f-ce1b-41b5-809b-327ba1ca0ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:47.748111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4934f-ce1b-41b5-809b-327ba1ca0ed3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:47.748603 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4934f-ce1b-41b5-809b-327ba1ca0ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:47.749096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4934f-ce1b-41b5-809b-327ba1ca0ed3 admin admin] PATH_INFO: `/v3/projects/7bea7e03e85645e49794b7236aa8ad79/users/d76063c239164f8aa736b3dc792b9c60/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:47.752788 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4934f-ce1b-41b5-809b-327ba1ca0ed3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bea7e03e85645e49794b7236aa8ad79, user_id=d76063c239164f8aa736b3dc792b9c60, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:47.756874 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4934f-ce1b-41b5-809b-327ba1ca0ed3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:47.777967 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1137/2270] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:47 2022] PUT /identity/v3/projects/7bea7e03e85645e49794b7236aa8ad79/users/d76063c239164f8aa736b3dc792b9c60/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:47.784264 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181d2e7-61b0-4493-97a0-a9e1aa1ae51e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:47.784561 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181d2e7-61b0-4493-97a0-a9e1aa1ae51e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:47.784794 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb5859-ab32-43c9-be80-ab09723982f0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:47.784912 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181d2e7-61b0-4493-97a0-a9e1aa1ae51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:47.821278 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4181d2e7-61b0-4493-97a0-a9e1aa1ae51e None None] MFA Rules not processed for user `d76063c239164f8aa736b3dc792b9c60`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:47.822737 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4181d2e7-61b0-4493-97a0-a9e1aa1ae51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:47.893213 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb5859-ab32-43c9-be80-ab09723982f0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:47.993811 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1138/2271] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:47 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:48.012657 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcfc245-f528-416c-b058-4ede9c6e8112 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:48.013324 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcfc245-f528-416c-b058-4ede9c6e8112 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:48.013623 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcfc245-f528-416c-b058-4ede9c6e8112 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:48.013905 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcfc245-f528-416c-b058-4ede9c6e8112 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:48.016985 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-afcfc245-f528-416c-b058-4ede9c6e8112 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:48.064643 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181d2e7-61b0-4493-97a0-a9e1aa1ae51e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:48.069562 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4181d2e7-61b0-4493-97a0-a9e1aa1ae51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:48.095129 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181d2e7-61b0-4493-97a0-a9e1aa1ae51e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:48.106958 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7573d9-d195-4a24-b781-a0bc91a360de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:48.107646 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7573d9-d195-4a24-b781-a0bc91a360de service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:48.107915 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7573d9-d195-4a24-b781-a0bc91a360de service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:48.108194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7573d9-d195-4a24-b781-a0bc91a360de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:48.131326 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7573d9-d195-4a24-b781-a0bc91a360de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:48.133154 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7573d9-d195-4a24-b781-a0bc91a360de service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:48.142255 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcfc245-f528-416c-b058-4ede9c6e8112 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:48.144047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcfc245-f528-416c-b058-4ede9c6e8112 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:48.147505 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1139/2272] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:48 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 16:05:48.160974 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1134/2273] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:47 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:48.382604 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7573d9-d195-4a24-b781-a0bc91a360de service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:48.396513 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7573d9-d195-4a24-b781-a0bc91a360de service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:48.401498 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff727397-dd2a-4175-833f-b06aeb5af1c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:48.402614 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff727397-dd2a-4175-833f-b06aeb5af1c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:48.403727 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff727397-dd2a-4175-833f-b06aeb5af1c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:48.404085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff727397-dd2a-4175-833f-b06aeb5af1c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:48.497317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:48.552501 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff727397-dd2a-4175-833f-b06aeb5af1c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:48.554790 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff727397-dd2a-4175-833f-b06aeb5af1c1 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:48.565856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1140/2274] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:48 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:50.071758 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01693edd-a40a-4a10-91e4-1ed11aef0d88 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:50.072302 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01693edd-a40a-4a10-91e4-1ed11aef0d88 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:50.072302 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01693edd-a40a-4a10-91e4-1ed11aef0d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:50.109657 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-01693edd-a40a-4a10-91e4-1ed11aef0d88 None None] MFA Rules not processed for user `57361619d5c34441b16ad617a4fe22d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:50.111427 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-01693edd-a40a-4a10-91e4-1ed11aef0d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:50.207592 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:50.220320 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1135/2275] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:50.230978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5702a-9be0-40f1-9b13-c0064212d4ab None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:50.231883 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5702a-9be0-40f1-9b13-c0064212d4ab None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:50.232321 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5702a-9be0-40f1-9b13-c0064212d4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:50.256708 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-25c5702a-9be0-40f1-9b13-c0064212d4ab None None] MFA Rules not processed for user `57361619d5c34441b16ad617a4fe22d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:50.257963 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-25c5702a-9be0-40f1-9b13-c0064212d4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:50.269997 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1141/2276] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:50.279229 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01693edd-a40a-4a10-91e4-1ed11aef0d88 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:50.281875 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-01693edd-a40a-4a10-91e4-1ed11aef0d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:50.397746 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01693edd-a40a-4a10-91e4-1ed11aef0d88 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:50.403815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154e123-72f0-4857-8964-541677ace273 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:50.404667 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154e123-72f0-4857-8964-541677ace273 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:50.405948 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154e123-72f0-4857-8964-541677ace273 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:50.406319 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154e123-72f0-4857-8964-541677ace273 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:50.418864 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154e123-72f0-4857-8964-541677ace273 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:50.420011 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154e123-72f0-4857-8964-541677ace273 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:50.431593 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1136/2277] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 16:05:51.330804 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8c77f-905b-41e9-a549-f47132bbfede None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:51.331270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8c77f-905b-41e9-a549-f47132bbfede None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:51.331522 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8c77f-905b-41e9-a549-f47132bbfede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:51.362680 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-04c8c77f-905b-41e9-a549-f47132bbfede None None] MFA Rules not processed for user `57361619d5c34441b16ad617a4fe22d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:51.363653 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-04c8c77f-905b-41e9-a549-f47132bbfede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:51.384537 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1142/2278] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:53.743079 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154e123-72f0-4857-8964-541677ace273 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:53.744509 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6154e123-72f0-4857-8964-541677ace273 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:53.887141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154e123-72f0-4857-8964-541677ace273 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:53.898123 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69329da-f06f-447a-b18a-8fbd4dbfeebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:53.902189 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69329da-f06f-447a-b18a-8fbd4dbfeebf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:53.902477 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69329da-f06f-447a-b18a-8fbd4dbfeebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:53.903215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69329da-f06f-447a-b18a-8fbd4dbfeebf admin admin] PATH_INFO: `/v3/users/280c7a2aae73450d82bf455810582627` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:53.905759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69329da-f06f-447a-b18a-8fbd4dbfeebf admin admin] RBAC: Authorizing `identity:delete_user(user_id=280c7a2aae73450d82bf455810582627)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:53.913169 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69329da-f06f-447a-b18a-8fbd4dbfeebf admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:54.019336 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a69329da-f06f-447a-b18a-8fbd4dbfeebf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '280c7a2aae73450d82bf455810582627'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:54.020875 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a69329da-f06f-447a-b18a-8fbd4dbfeebf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '280c7a2aae73450d82bf455810582627'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:54.021791 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a69329da-f06f-447a-b18a-8fbd4dbfeebf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '280c7a2aae73450d82bf455810582627'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:54.032266 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a69329da-f06f-447a-b18a-8fbd4dbfeebf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '280c7a2aae73450d82bf455810582627'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:54.043087 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a69329da-f06f-447a-b18a-8fbd4dbfeebf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '280c7a2aae73450d82bf455810582627'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:54.063549 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:54.073408 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1137/2279] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:53 2022] DELETE /identity/v3/users/280c7a2aae73450d82bf455810582627 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:54.078992 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8ccee-7c12-487c-aba2-79abd6007231 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:54.079458 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8ccee-7c12-487c-aba2-79abd6007231 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:54.079565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8ccee-7c12-487c-aba2-79abd6007231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:54.106823 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0ba8ccee-7c12-487c-aba2-79abd6007231 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:54.108176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0ba8ccee-7c12-487c-aba2-79abd6007231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:54.181671 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1143/2280] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:54.189516 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69329da-f06f-447a-b18a-8fbd4dbfeebf admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:54.191326 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a69329da-f06f-447a-b18a-8fbd4dbfeebf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:54.275719 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69329da-f06f-447a-b18a-8fbd4dbfeebf admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:54.284532 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55314302-d512-4987-8bd1-1155a97d5640 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:54.285218 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55314302-d512-4987-8bd1-1155a97d5640 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:54.286275 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55314302-d512-4987-8bd1-1155a97d5640 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:54.287106 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55314302-d512-4987-8bd1-1155a97d5640 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:54.310648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55314302-d512-4987-8bd1-1155a97d5640 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:54.311790 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55314302-d512-4987-8bd1-1155a97d5640 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:54.324184 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1138/2281] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:55.168418 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8ccee-7c12-487c-aba2-79abd6007231 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:55.169930 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0ba8ccee-7c12-487c-aba2-79abd6007231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:55.192672 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8ccee-7c12-487c-aba2-79abd6007231 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:55.201694 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6bbf13-ca0d-4579-8813-f14aae81ce3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:55.210181 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6bbf13-ca0d-4579-8813-f14aae81ce3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:55.210472 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6bbf13-ca0d-4579-8813-f14aae81ce3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:55.210785 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6bbf13-ca0d-4579-8813-f14aae81ce3d admin admin] PATH_INFO: `/v3/projects/7acfd0b52e544c59a62e7d15e94d99ec` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:55.212625 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6bbf13-ca0d-4579-8813-f14aae81ce3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7acfd0b52e544c59a62e7d15e94d99ec)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:55.214803 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6bbf13-ca0d-4579-8813-f14aae81ce3d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:55.280763 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7b6bbf13-ca0d-4579-8813-f14aae81ce3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7acfd0b52e544c59a62e7d15e94d99ec', 'request_id': 'req-7b6bbf13-ca0d-4579-8813-f14aae81ce3d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:55.289017 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7b6bbf13-ca0d-4579-8813-f14aae81ce3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7acfd0b52e544c59a62e7d15e94d99ec', 'request_id': 'req-7b6bbf13-ca0d-4579-8813-f14aae81ce3d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:55.297629 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7b6bbf13-ca0d-4579-8813-f14aae81ce3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7acfd0b52e544c59a62e7d15e94d99ec', 'request_id': 'req-7b6bbf13-ca0d-4579-8813-f14aae81ce3d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:55.320730 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7b6bbf13-ca0d-4579-8813-f14aae81ce3d admin admin] The token cache is being invalidate because project 7acfd0b52e544c59a62e7d15e94d99ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:55.321178 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7b6bbf13-ca0d-4579-8813-f14aae81ce3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:55.321959 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7b6bbf13-ca0d-4579-8813-f14aae81ce3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:55.324347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1144/2282] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:55 2022] DELETE /identity/v3/projects/7acfd0b52e544c59a62e7d15e94d99ec => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:55.328192 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55314302-d512-4987-8bd1-1155a97d5640 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:55.341405 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55314302-d512-4987-8bd1-1155a97d5640 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:55.434324 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e579e-b7c6-4c1a-a5f5-dfb59ec7fec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:55.436536 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e579e-b7c6-4c1a-a5f5-dfb59ec7fec0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:55.437850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e579e-b7c6-4c1a-a5f5-dfb59ec7fec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:55.438535 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e579e-b7c6-4c1a-a5f5-dfb59ec7fec0 admin admin] PATH_INFO: `/v3/users/ae651794b182425d8ebc68e6740bcfc0` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:55.440235 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209e579e-b7c6-4c1a-a5f5-dfb59ec7fec0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae651794b182425d8ebc68e6740bcfc0)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:55.442651 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209e579e-b7c6-4c1a-a5f5-dfb59ec7fec0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:55.524654 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-209e579e-b7c6-4c1a-a5f5-dfb59ec7fec0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae651794b182425d8ebc68e6740bcfc0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:55.525612 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-209e579e-b7c6-4c1a-a5f5-dfb59ec7fec0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae651794b182425d8ebc68e6740bcfc0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:55.526376 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-209e579e-b7c6-4c1a-a5f5-dfb59ec7fec0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae651794b182425d8ebc68e6740bcfc0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:55.538845 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-209e579e-b7c6-4c1a-a5f5-dfb59ec7fec0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae651794b182425d8ebc68e6740bcfc0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:55.548385 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-209e579e-b7c6-4c1a-a5f5-dfb59ec7fec0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae651794b182425d8ebc68e6740bcfc0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:55.569241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1139/2283] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:55 2022] DELETE /identity/v3/users/ae651794b182425d8ebc68e6740bcfc0 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:56.154063 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6bbf13-ca0d-4579-8813-f14aae81ce3d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:56.155677 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7b6bbf13-ca0d-4579-8813-f14aae81ce3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:56.171063 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6bbf13-ca0d-4579-8813-f14aae81ce3d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:56.211639 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:56.250719 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b439374-0a7e-47a6-b436-c25d3be8231d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:56.251872 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b439374-0a7e-47a6-b436-c25d3be8231d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:56.252257 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b439374-0a7e-47a6-b436-c25d3be8231d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:56.252662 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b439374-0a7e-47a6-b436-c25d3be8231d admin admin] PATH_INFO: `/v3/projects/434be561fe904fdc8a3eb1d7562d404b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:56.254175 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b439374-0a7e-47a6-b436-c25d3be8231d admin admin] RBAC: Authorizing `identity:delete_project(project_id=434be561fe904fdc8a3eb1d7562d404b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:56.256921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b439374-0a7e-47a6-b436-c25d3be8231d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:56.312763 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3b439374-0a7e-47a6-b436-c25d3be8231d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '434be561fe904fdc8a3eb1d7562d404b', 'request_id': 'req-3b439374-0a7e-47a6-b436-c25d3be8231d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:56.321810 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3b439374-0a7e-47a6-b436-c25d3be8231d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '434be561fe904fdc8a3eb1d7562d404b', 'request_id': 'req-3b439374-0a7e-47a6-b436-c25d3be8231d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:56.328316 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3b439374-0a7e-47a6-b436-c25d3be8231d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '434be561fe904fdc8a3eb1d7562d404b', 'request_id': 'req-3b439374-0a7e-47a6-b436-c25d3be8231d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:56.340647 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3b439374-0a7e-47a6-b436-c25d3be8231d admin admin] The token cache is being invalidate because project 434be561fe904fdc8a3eb1d7562d404b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:56.340974 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3b439374-0a7e-47a6-b436-c25d3be8231d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:56.342278 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3b439374-0a7e-47a6-b436-c25d3be8231d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:56.344172 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1145/2284] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:56 2022] DELETE /identity/v3/projects/434be561fe904fdc8a3eb1d7562d404b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:56.350701 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e579e-b7c6-4c1a-a5f5-dfb59ec7fec0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:56.365476 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-209e579e-b7c6-4c1a-a5f5-dfb59ec7fec0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:56.397861 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e579e-b7c6-4c1a-a5f5-dfb59ec7fec0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:56.421825 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:56.468714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d0ef1-34bc-4690-a5d3-11159452b8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:56.469620 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d0ef1-34bc-4690-a5d3-11159452b8af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:56.469904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d0ef1-34bc-4690-a5d3-11159452b8af admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:56.470208 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d0ef1-34bc-4690-a5d3-11159452b8af admin admin] PATH_INFO: `/v3/users/f14b1aa7cc7245f59312350c29adcea8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:56.471473 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2d0ef1-34bc-4690-a5d3-11159452b8af admin admin] RBAC: Authorizing `identity:delete_user(user_id=f14b1aa7cc7245f59312350c29adcea8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:56.473750 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2d0ef1-34bc-4690-a5d3-11159452b8af admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:56.534435 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1d2d0ef1-34bc-4690-a5d3-11159452b8af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f14b1aa7cc7245f59312350c29adcea8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:56.536168 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1d2d0ef1-34bc-4690-a5d3-11159452b8af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f14b1aa7cc7245f59312350c29adcea8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:56.537507 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1d2d0ef1-34bc-4690-a5d3-11159452b8af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f14b1aa7cc7245f59312350c29adcea8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:56.552383 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1d2d0ef1-34bc-4690-a5d3-11159452b8af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f14b1aa7cc7245f59312350c29adcea8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:56.571033 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1d2d0ef1-34bc-4690-a5d3-11159452b8af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f14b1aa7cc7245f59312350c29adcea8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:56.592796 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1140/2285] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:56 2022] DELETE /identity/v3/users/f14b1aa7cc7245f59312350c29adcea8 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:56.784367 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b439374-0a7e-47a6-b436-c25d3be8231d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:56.786229 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3b439374-0a7e-47a6-b436-c25d3be8231d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:56.804460 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b439374-0a7e-47a6-b436-c25d3be8231d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:56.885862 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7343a6d7-a071-4639-87bb-888f12b98d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:56.887153 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7343a6d7-a071-4639-87bb-888f12b98d00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:56.887562 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7343a6d7-a071-4639-87bb-888f12b98d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:56.887940 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7343a6d7-a071-4639-87bb-888f12b98d00 admin admin] PATH_INFO: `/v3/users/213ff2b8cf36404fb4cf86d13463f6e8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:56.889277 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7343a6d7-a071-4639-87bb-888f12b98d00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=213ff2b8cf36404fb4cf86d13463f6e8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:56.891559 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7343a6d7-a071-4639-87bb-888f12b98d00 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:56.958616 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7343a6d7-a071-4639-87bb-888f12b98d00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '213ff2b8cf36404fb4cf86d13463f6e8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:56.959882 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7343a6d7-a071-4639-87bb-888f12b98d00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '213ff2b8cf36404fb4cf86d13463f6e8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:56.960544 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7343a6d7-a071-4639-87bb-888f12b98d00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '213ff2b8cf36404fb4cf86d13463f6e8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:56.969512 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7343a6d7-a071-4639-87bb-888f12b98d00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '213ff2b8cf36404fb4cf86d13463f6e8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:56.982241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d0ef1-34bc-4690-a5d3-11159452b8af admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:56.983889 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1d2d0ef1-34bc-4690-a5d3-11159452b8af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:56.992526 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7343a6d7-a071-4639-87bb-888f12b98d00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '213ff2b8cf36404fb4cf86d13463f6e8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:56.995414 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:57.006925 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d0ef1-34bc-4690-a5d3-11159452b8af admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:57.021160 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a3141-07fc-46d2-9d85-1318ee55b027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:57.022100 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a3141-07fc-46d2-9d85-1318ee55b027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:57.022389 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a3141-07fc-46d2-9d85-1318ee55b027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:57.022671 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a3141-07fc-46d2-9d85-1318ee55b027 admin admin] PATH_INFO: `/v3/projects/b46d01a124c94769bda9d27bab07b657` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:57.026363 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a3141-07fc-46d2-9d85-1318ee55b027 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b46d01a124c94769bda9d27bab07b657)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:57.028431 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a3141-07fc-46d2-9d85-1318ee55b027 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:57.037546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1146/2286] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:56 2022] DELETE /identity/v3/users/213ff2b8cf36404fb4cf86d13463f6e8 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:57.041816 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90abb66-6dd3-4c35-829a-f0481c10d38d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:57.042555 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90abb66-6dd3-4c35-829a-f0481c10d38d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:57.042555 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90abb66-6dd3-4c35-829a-f0481c10d38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:57.068459 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a90abb66-6dd3-4c35-829a-f0481c10d38d None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:57.071110 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a90abb66-6dd3-4c35-829a-f0481c10d38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:57.125275 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ce4a3141-07fc-46d2-9d85-1318ee55b027 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b46d01a124c94769bda9d27bab07b657', 'request_id': 'req-ce4a3141-07fc-46d2-9d85-1318ee55b027'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:57.135473 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ce4a3141-07fc-46d2-9d85-1318ee55b027 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b46d01a124c94769bda9d27bab07b657', 'request_id': 'req-ce4a3141-07fc-46d2-9d85-1318ee55b027'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:57.152817 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ce4a3141-07fc-46d2-9d85-1318ee55b027 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b46d01a124c94769bda9d27bab07b657', 'request_id': 'req-ce4a3141-07fc-46d2-9d85-1318ee55b027'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:57.173048 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ce4a3141-07fc-46d2-9d85-1318ee55b027 admin admin] The token cache is being invalidate because project b46d01a124c94769bda9d27bab07b657 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:57.174097 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ce4a3141-07fc-46d2-9d85-1318ee55b027 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:57.175282 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ce4a3141-07fc-46d2-9d85-1318ee55b027 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:57.179185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1141/2287] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:56 2022] DELETE /identity/v3/projects/b46d01a124c94769bda9d27bab07b657 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:57.196758 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45de26f-80e1-4f2d-bcdc-4795cc96ccbc None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:57.198431 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45de26f-80e1-4f2d-bcdc-4795cc96ccbc None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:57.199216 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45de26f-80e1-4f2d-bcdc-4795cc96ccbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:57.206062 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1147/2288] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:05:57.234586 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e45de26f-80e1-4f2d-bcdc-4795cc96ccbc None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:57.236328 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e45de26f-80e1-4f2d-bcdc-4795cc96ccbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:57.252553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90abb66-6dd3-4c35-829a-f0481c10d38d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:57.256659 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a90abb66-6dd3-4c35-829a-f0481c10d38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:57.286533 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:57.391762 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1142/2289] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:05:57.393716 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45de26f-80e1-4f2d-bcdc-4795cc96ccbc None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:57.401362 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e45de26f-80e1-4f2d-bcdc-4795cc96ccbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:57.424059 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45de26f-80e1-4f2d-bcdc-4795cc96ccbc None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:57.438940 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90abb66-6dd3-4c35-829a-f0481c10d38d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:57.441940 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b9812-cef9-49cc-ac82-b90bb28c5c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:57.443252 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b9812-cef9-49cc-ac82-b90bb28c5c22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:57.444032 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b9812-cef9-49cc-ac82-b90bb28c5c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:57.444321 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b9812-cef9-49cc-ac82-b90bb28c5c22 admin admin] PATH_INFO: `/v3/users/6a966c72cd384d379802a23407abdebb` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:57.446882 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507b9812-cef9-49cc-ac82-b90bb28c5c22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a966c72cd384d379802a23407abdebb)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:57.449378 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507b9812-cef9-49cc-ac82-b90bb28c5c22 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:57.540429 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-507b9812-cef9-49cc-ac82-b90bb28c5c22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a966c72cd384d379802a23407abdebb'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:57.541094 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-507b9812-cef9-49cc-ac82-b90bb28c5c22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a966c72cd384d379802a23407abdebb'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:57.541903 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-507b9812-cef9-49cc-ac82-b90bb28c5c22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a966c72cd384d379802a23407abdebb'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:57.556528 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-507b9812-cef9-49cc-ac82-b90bb28c5c22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a966c72cd384d379802a23407abdebb'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:57.557285 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3fd30-84b4-463d-93b4-7232ba05a321 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:57.558741 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3fd30-84b4-463d-93b4-7232ba05a321 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:57.559149 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3fd30-84b4-463d-93b4-7232ba05a321 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:57.559521 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3fd30-84b4-463d-93b4-7232ba05a321 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:57.564088 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-61a3fd30-84b4-463d-93b4-7232ba05a321 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:57.571305 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-507b9812-cef9-49cc-ac82-b90bb28c5c22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a966c72cd384d379802a23407abdebb'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:57.586615 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a3fd30-84b4-463d-93b4-7232ba05a321 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:57.588339 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a3fd30-84b4-463d-93b4-7232ba05a321 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:57.614213 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1148/2290] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:57.632591 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1143/2291] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:57 2022] DELETE /identity/v3/users/6a966c72cd384d379802a23407abdebb => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:57.636588 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd6658-d6e5-4f09-bb0f-99447ee795dd None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:57.636878 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd6658-d6e5-4f09-bb0f-99447ee795dd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:57.637163 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd6658-d6e5-4f09-bb0f-99447ee795dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:57.638443 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57dd58e-112a-497f-a24f-dfd8157b6a9d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:57.640254 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57dd58e-112a-497f-a24f-dfd8157b6a9d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:57.640655 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57dd58e-112a-497f-a24f-dfd8157b6a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:57.666422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ecdd6658-d6e5-4f09-bb0f-99447ee795dd None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:57.667730 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ecdd6658-d6e5-4f09-bb0f-99447ee795dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:57.686791 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e57dd58e-112a-497f-a24f-dfd8157b6a9d None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:57.688345 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e57dd58e-112a-497f-a24f-dfd8157b6a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:57.743477 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:57.794802 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1144/2292] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:05:57.802999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1149/2293] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:57.808092 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57dd58e-112a-497f-a24f-dfd8157b6a9d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:57.816522 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd6658-d6e5-4f09-bb0f-99447ee795dd None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:57.820613 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57dd58e-112a-497f-a24f-dfd8157b6a9d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:57.822708 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ecdd6658-d6e5-4f09-bb0f-99447ee795dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:57.825728 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75841e6-8e96-4a0b-a6e3-abd9096a15ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:57.826482 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75841e6-8e96-4a0b-a6e3-abd9096a15ee admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:57.826841 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75841e6-8e96-4a0b-a6e3-abd9096a15ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:57.827188 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75841e6-8e96-4a0b-a6e3-abd9096a15ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:57.829202 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75841e6-8e96-4a0b-a6e3-abd9096a15ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:05:57.829202 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75841e6-8e96-4a0b-a6e3-abd9096a15ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:57.832385 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75841e6-8e96-4a0b-a6e3-abd9096a15ee admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:57.850012 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1145/2294] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:05:57.851587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75841e6-8e96-4a0b-a6e3-abd9096a15ee admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:57.862927 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75841e6-8e96-4a0b-a6e3-abd9096a15ee admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:57.867935 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a3ffd-3236-4b19-9a87-8fa31faf387d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:57.868605 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a3ffd-3236-4b19-9a87-8fa31faf387d admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:57.868891 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a3ffd-3236-4b19-9a87-8fa31faf387d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:57.869168 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a3ffd-3236-4b19-9a87-8fa31faf387d admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:57.870094 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a3ffd-3236-4b19-9a87-8fa31faf387d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:57.872518 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a3ffd-3236-4b19-9a87-8fa31faf387d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:57.940629 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1146/2295] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:57 2022] POST /identity/v3/projects => generated 385 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:05:57.946254 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a3ffd-3236-4b19-9a87-8fa31faf387d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:57.976817 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:57.981122 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd6658-d6e5-4f09-bb0f-99447ee795dd None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:58.059457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a3ffd-3236-4b19-9a87-8fa31faf387d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:58.065257 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cfc4d0-7655-4707-946e-b968aaf4b059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:58.066096 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfc4d0-7655-4707-946e-b968aaf4b059 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:58.066614 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfc4d0-7655-4707-946e-b968aaf4b059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:58.066968 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfc4d0-7655-4707-946e-b968aaf4b059 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:58.068899 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cfc4d0-7655-4707-946e-b968aaf4b059 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:58.071744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cfc4d0-7655-4707-946e-b968aaf4b059 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:58.096245 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0b92bf-625d-4c35-a622-dc35dea68a87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:58.096913 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b92bf-625d-4c35-a622-dc35dea68a87 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:58.097198 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b92bf-625d-4c35-a622-dc35dea68a87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:58.097479 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b92bf-625d-4c35-a622-dc35dea68a87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:58.120014 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1147/2296] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:57 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:58.125896 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cfc4d0-7655-4707-946e-b968aaf4b059 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:58.127106 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0b92bf-625d-4c35-a622-dc35dea68a87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:58.128585 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0b92bf-625d-4c35-a622-dc35dea68a87 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:58.140244 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1150/2297] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:05:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:05:58.156916 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cfc4d0-7655-4707-946e-b968aaf4b059 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:58.166066 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f569f33d-7b26-42ff-97f7-1f0e322d0951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:58.167107 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569f33d-7b26-42ff-97f7-1f0e322d0951 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:58.168538 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569f33d-7b26-42ff-97f7-1f0e322d0951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:58.168914 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569f33d-7b26-42ff-97f7-1f0e322d0951 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:58.169802 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f569f33d-7b26-42ff-97f7-1f0e322d0951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:58.171581 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f569f33d-7b26-42ff-97f7-1f0e322d0951 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:58.191071 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1148/2298] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:58.192327 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0b92bf-625d-4c35-a622-dc35dea68a87 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:58.205534 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0b92bf-625d-4c35-a622-dc35dea68a87 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:58.215226 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ca33a-31e0-4f64-8595-af2acde81826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:58.216040 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ca33a-31e0-4f64-8595-af2acde81826 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:58.216882 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ca33a-31e0-4f64-8595-af2acde81826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:58.217267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ca33a-31e0-4f64-8595-af2acde81826 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:58.218817 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421ca33a-31e0-4f64-8595-af2acde81826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:58.221315 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421ca33a-31e0-4f64-8595-af2acde81826 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:58.242092 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f569f33d-7b26-42ff-97f7-1f0e322d0951 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:58.255890 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f569f33d-7b26-42ff-97f7-1f0e322d0951 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:58.264005 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1151/2299] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:58.268144 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ca33a-31e0-4f64-8595-af2acde81826 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:58.278674 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f569f33d-7b26-42ff-97f7-1f0e322d0951 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:58.280137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ca33a-31e0-4f64-8595-af2acde81826 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:58.284213 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3169b7c-d4ac-4c95-b3bf-ff4b20e4694f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:58.285474 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3169b7c-d4ac-4c95-b3bf-ff4b20e4694f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:58.285619 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4997154b-9bfb-4309-b255-4e92e8016718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:58.285745 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3169b7c-d4ac-4c95-b3bf-ff4b20e4694f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:58.286090 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3169b7c-d4ac-4c95-b3bf-ff4b20e4694f admin admin] PATH_INFO: `/v3/projects/3ca593f4cc714fa6a49cc55d8d3e91ad` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:58.286797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997154b-9bfb-4309-b255-4e92e8016718 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:58.287125 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997154b-9bfb-4309-b255-4e92e8016718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:58.287455 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997154b-9bfb-4309-b255-4e92e8016718 admin admin] PATH_INFO: `/v3/projects/ee3300f56e7346ee8ecc93683e200c48/users/f1e915375ff14eb1a54fef6a9cc8e4a0/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:58.287921 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3169b7c-d4ac-4c95-b3bf-ff4b20e4694f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ca593f4cc714fa6a49cc55d8d3e91ad)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:58.290211 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3169b7c-d4ac-4c95-b3bf-ff4b20e4694f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:58.317860 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4997154b-9bfb-4309-b255-4e92e8016718 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee3300f56e7346ee8ecc93683e200c48, user_id=f1e915375ff14eb1a54fef6a9cc8e4a0, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:58.321460 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4997154b-9bfb-4309-b255-4e92e8016718 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:58.342570 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1152/2300] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:58 2022] PUT /identity/v3/projects/ee3300f56e7346ee8ecc93683e200c48/users/f1e915375ff14eb1a54fef6a9cc8e4a0/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:58.357080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318869c-f19b-44fa-9bfe-d7e5cd7ed919 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:58.357080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318869c-f19b-44fa-9bfe-d7e5cd7ed919 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:58.357080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318869c-f19b-44fa-9bfe-d7e5cd7ed919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:58.387893 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c3169b7c-d4ac-4c95-b3bf-ff4b20e4694f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ca593f4cc714fa6a49cc55d8d3e91ad', 'request_id': 'req-c3169b7c-d4ac-4c95-b3bf-ff4b20e4694f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:58.399967 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c3169b7c-d4ac-4c95-b3bf-ff4b20e4694f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ca593f4cc714fa6a49cc55d8d3e91ad', 'request_id': 'req-c3169b7c-d4ac-4c95-b3bf-ff4b20e4694f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:58.406085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1318869c-f19b-44fa-9bfe-d7e5cd7ed919 None None] MFA Rules not processed for user `f1e915375ff14eb1a54fef6a9cc8e4a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:58.407319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1318869c-f19b-44fa-9bfe-d7e5cd7ed919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:58.431258 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c3169b7c-d4ac-4c95-b3bf-ff4b20e4694f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ca593f4cc714fa6a49cc55d8d3e91ad', 'request_id': 'req-c3169b7c-d4ac-4c95-b3bf-ff4b20e4694f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:58.447979 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c3169b7c-d4ac-4c95-b3bf-ff4b20e4694f admin admin] The token cache is being invalidate because project 3ca593f4cc714fa6a49cc55d8d3e91ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:58.448248 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c3169b7c-d4ac-4c95-b3bf-ff4b20e4694f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:58.449440 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c3169b7c-d4ac-4c95-b3bf-ff4b20e4694f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:58.451218 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1149/2301] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:58 2022] DELETE /identity/v3/projects/3ca593f4cc714fa6a49cc55d8d3e91ad => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:58.459852 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3169b7c-d4ac-4c95-b3bf-ff4b20e4694f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:58.461323 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c3169b7c-d4ac-4c95-b3bf-ff4b20e4694f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:58.518414 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:58.530573 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1153/2302] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:58.535192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1318869c-f19b-44fa-9bfe-d7e5cd7ed919 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:58.537565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1318869c-f19b-44fa-9bfe-d7e5cd7ed919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:58.548883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3169b7c-d4ac-4c95-b3bf-ff4b20e4694f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:58.555923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398bad65-e8bd-4b23-8f83-90304177133f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:58.557158 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398bad65-e8bd-4b23-8f83-90304177133f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:58.557803 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398bad65-e8bd-4b23-8f83-90304177133f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:58.558312 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398bad65-e8bd-4b23-8f83-90304177133f admin admin] PATH_INFO: `/v3/users/d76063c239164f8aa736b3dc792b9c60` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:58.559771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398bad65-e8bd-4b23-8f83-90304177133f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d76063c239164f8aa736b3dc792b9c60)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:58.561995 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398bad65-e8bd-4b23-8f83-90304177133f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:58.567566 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:58.574695 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1318869c-f19b-44fa-9bfe-d7e5cd7ed919 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:58.584384 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a2bfd4-13dc-413d-b140-2121c955cd61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:58.585260 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a2bfd4-13dc-413d-b140-2121c955cd61 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:58.585654 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a2bfd4-13dc-413d-b140-2121c955cd61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:58.586004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a2bfd4-13dc-413d-b140-2121c955cd61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:58.586798 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a2bfd4-13dc-413d-b140-2121c955cd61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:58.595173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a2bfd4-13dc-413d-b140-2121c955cd61 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:58.644230 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-398bad65-e8bd-4b23-8f83-90304177133f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd76063c239164f8aa736b3dc792b9c60'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:58.646625 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-398bad65-e8bd-4b23-8f83-90304177133f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd76063c239164f8aa736b3dc792b9c60'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:58.648690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1154/2303] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:58 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:58.648939 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-398bad65-e8bd-4b23-8f83-90304177133f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd76063c239164f8aa736b3dc792b9c60'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:58.652589 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a2bfd4-13dc-413d-b140-2121c955cd61 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:58.654767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d8a2bfd4-13dc-413d-b140-2121c955cd61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:58.659160 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-398bad65-e8bd-4b23-8f83-90304177133f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd76063c239164f8aa736b3dc792b9c60'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:58.670955 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-398bad65-e8bd-4b23-8f83-90304177133f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd76063c239164f8aa736b3dc792b9c60'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:58.715163 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1150/2304] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:05:58 2022] DELETE /identity/v3/users/d76063c239164f8aa736b3dc792b9c60 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:58.732977 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398bad65-e8bd-4b23-8f83-90304177133f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:58.734871 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-398bad65-e8bd-4b23-8f83-90304177133f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:58.756970 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a2bfd4-13dc-413d-b140-2121c955cd61 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:58.829349 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398bad65-e8bd-4b23-8f83-90304177133f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:58.835620 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52245c6f-4762-41df-bcc1-2ea289ac464f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:58.836688 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52245c6f-4762-41df-bcc1-2ea289ac464f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:58.837062 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52245c6f-4762-41df-bcc1-2ea289ac464f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:58.837374 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52245c6f-4762-41df-bcc1-2ea289ac464f admin admin] PATH_INFO: `/v3/projects/4d1c1ee916984d4f86e3c7c3543b8154` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:58.838223 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ee06f-4bbe-4a2e-8260-e6c7a7c69858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:58.838665 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52245c6f-4762-41df-bcc1-2ea289ac464f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d1c1ee916984d4f86e3c7c3543b8154)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:58.838911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ee06f-4bbe-4a2e-8260-e6c7a7c69858 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:58.839200 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ee06f-4bbe-4a2e-8260-e6c7a7c69858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:58.839485 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ee06f-4bbe-4a2e-8260-e6c7a7c69858 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:58.840360 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ee06f-4bbe-4a2e-8260-e6c7a7c69858 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:58.840774 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52245c6f-4762-41df-bcc1-2ea289ac464f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:58.842475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ee06f-4bbe-4a2e-8260-e6c7a7c69858 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:58.868203 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1155/2305] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:58 2022] POST /identity/v3/users => generated 293 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:05:58.869448 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ee06f-4bbe-4a2e-8260-e6c7a7c69858 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:58.883133 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ee06f-4bbe-4a2e-8260-e6c7a7c69858 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:58.918280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-52245c6f-4762-41df-bcc1-2ea289ac464f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d1c1ee916984d4f86e3c7c3543b8154', 'request_id': 'req-52245c6f-4762-41df-bcc1-2ea289ac464f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:58.934541 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-52245c6f-4762-41df-bcc1-2ea289ac464f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d1c1ee916984d4f86e3c7c3543b8154', 'request_id': 'req-52245c6f-4762-41df-bcc1-2ea289ac464f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:58.946108 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-52245c6f-4762-41df-bcc1-2ea289ac464f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d1c1ee916984d4f86e3c7c3543b8154', 'request_id': 'req-52245c6f-4762-41df-bcc1-2ea289ac464f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:58.961280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-52245c6f-4762-41df-bcc1-2ea289ac464f admin admin] The token cache is being invalidate because project 4d1c1ee916984d4f86e3c7c3543b8154 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:58.961840 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-52245c6f-4762-41df-bcc1-2ea289ac464f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:58.962844 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-52245c6f-4762-41df-bcc1-2ea289ac464f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:58.969675 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1151/2306] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:58 2022] DELETE /identity/v3/projects/4d1c1ee916984d4f86e3c7c3543b8154 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:58.976923 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd477e-3a57-4b0a-ba0b-f7492bb29f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:58.977766 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd477e-3a57-4b0a-ba0b-f7492bb29f4f admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:58.978686 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd477e-3a57-4b0a-ba0b-f7492bb29f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:58.978686 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd477e-3a57-4b0a-ba0b-f7492bb29f4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:58.979236 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd477e-3a57-4b0a-ba0b-f7492bb29f4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:58.981004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd477e-3a57-4b0a-ba0b-f7492bb29f4f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:58.996271 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1156/2307] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:59.001286 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52245c6f-4762-41df-bcc1-2ea289ac464f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:59.002941 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-52245c6f-4762-41df-bcc1-2ea289ac464f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:59.017866 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52245c6f-4762-41df-bcc1-2ea289ac464f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:59.024585 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bad14e-6c63-4ac9-bf43-d04ea3b01ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:59.025287 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bad14e-6c63-4ac9-bf43-d04ea3b01ce9 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:59.026205 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bad14e-6c63-4ac9-bf43-d04ea3b01ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:59.026205 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bad14e-6c63-4ac9-bf43-d04ea3b01ce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:59.027164 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bad14e-6c63-4ac9-bf43-d04ea3b01ce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:59.028916 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bad14e-6c63-4ac9-bf43-d04ea3b01ce9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:59.041613 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1152/2308] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:59.045478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd477e-3a57-4b0a-ba0b-f7492bb29f4f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:59.058526 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd477e-3a57-4b0a-ba0b-f7492bb29f4f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:59.066006 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f2af3-b388-4b50-9caa-3a4a5e895c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:59.067017 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f2af3-b388-4b50-9caa-3a4a5e895c2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:59.067347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f2af3-b388-4b50-9caa-3a4a5e895c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:59.067675 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f2af3-b388-4b50-9caa-3a4a5e895c2e admin admin] PATH_INFO: `/v3/projects/a8668dd5a9504d37ba3748324ade9beb/users/bfceef4921f84a928c66f5990ed19388/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:59.070626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:59.091325 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bad14e-6c63-4ac9-bf43-d04ea3b01ce9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:59.093493 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d7bad14e-6c63-4ac9-bf43-d04ea3b01ce9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:59.104092 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:59.107439 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f2af3-b388-4b50-9caa-3a4a5e895c2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8668dd5a9504d37ba3748324ade9beb, user_id=bfceef4921f84a928c66f5990ed19388, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:59.110727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bad14e-6c63-4ac9-bf43-d04ea3b01ce9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:59.111488 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f2af3-b388-4b50-9caa-3a4a5e895c2e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:59.118905 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c80f8c-8547-4d1e-a7af-7ba1578d3ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:59.121478 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c80f8c-8547-4d1e-a7af-7ba1578d3ffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:59.121853 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c80f8c-8547-4d1e-a7af-7ba1578d3ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:59.122566 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c80f8c-8547-4d1e-a7af-7ba1578d3ffd admin admin] PATH_INFO: `/v3/projects/7bea7e03e85645e49794b7236aa8ad79` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:59.124966 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c80f8c-8547-4d1e-a7af-7ba1578d3ffd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bea7e03e85645e49794b7236aa8ad79)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:59.130669 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c80f8c-8547-4d1e-a7af-7ba1578d3ffd admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:59.138054 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1157/2309] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:59 2022] PUT /identity/v3/projects/a8668dd5a9504d37ba3748324ade9beb/users/bfceef4921f84a928c66f5990ed19388/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:05:59.205948 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f0c80f8c-8547-4d1e-a7af-7ba1578d3ffd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bea7e03e85645e49794b7236aa8ad79', 'request_id': 'req-f0c80f8c-8547-4d1e-a7af-7ba1578d3ffd'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:59.215521 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f0c80f8c-8547-4d1e-a7af-7ba1578d3ffd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bea7e03e85645e49794b7236aa8ad79', 'request_id': 'req-f0c80f8c-8547-4d1e-a7af-7ba1578d3ffd'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:59.230312 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae875cf0-42ac-452f-84d4-dc66f42f88fb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:59.230878 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae875cf0-42ac-452f-84d4-dc66f42f88fb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:59.231084 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae875cf0-42ac-452f-84d4-dc66f42f88fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:59.231615 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f0c80f8c-8547-4d1e-a7af-7ba1578d3ffd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bea7e03e85645e49794b7236aa8ad79', 'request_id': 'req-f0c80f8c-8547-4d1e-a7af-7ba1578d3ffd'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:59.253487 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f0c80f8c-8547-4d1e-a7af-7ba1578d3ffd admin admin] The token cache is being invalidate because project 7bea7e03e85645e49794b7236aa8ad79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:05:59.253925 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f0c80f8c-8547-4d1e-a7af-7ba1578d3ffd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:59.254721 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f0c80f8c-8547-4d1e-a7af-7ba1578d3ffd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:05:59.256702 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1153/2310] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:05:59 2022] DELETE /identity/v3/projects/7bea7e03e85645e49794b7236aa8ad79 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:59.262451 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf68f4d-60a9-49d2-a9b0-5b77e9f0a898 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:59.263102 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf68f4d-60a9-49d2-a9b0-5b77e9f0a898 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:59.263630 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf68f4d-60a9-49d2-a9b0-5b77e9f0a898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:59.274491 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ae875cf0-42ac-452f-84d4-dc66f42f88fb None None] MFA Rules not processed for user `bfceef4921f84a928c66f5990ed19388`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:59.275779 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ae875cf0-42ac-452f-84d4-dc66f42f88fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:59.289305 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6bf68f4d-60a9-49d2-a9b0-5b77e9f0a898 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:59.290873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6bf68f4d-60a9-49d2-a9b0-5b77e9f0a898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:59.382112 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1158/2311] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:05:59.386321 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae875cf0-42ac-452f-84d4-dc66f42f88fb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:59.387905 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ae875cf0-42ac-452f-84d4-dc66f42f88fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:59.409891 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1154/2312] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:05:59.412165 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6b4da-92bf-4eb9-bd98-e3cb4d37b623 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:59.412471 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6b4da-92bf-4eb9-bd98-e3cb4d37b623 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:59.412759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6b4da-92bf-4eb9-bd98-e3cb4d37b623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:59.434459 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-38a6b4da-92bf-4eb9-bd98-e3cb4d37b623 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:05:59.435614 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-38a6b4da-92bf-4eb9-bd98-e3cb4d37b623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:05:59.448333 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1155/2313] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:05:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:05:59.449471 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6b4da-92bf-4eb9-bd98-e3cb4d37b623 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:59.459801 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6b4da-92bf-4eb9-bd98-e3cb4d37b623 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:59.465133 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a64a3-fd8a-499c-9e1b-7bf70ff7dd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:59.465848 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a64a3-fd8a-499c-9e1b-7bf70ff7dd6c admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:59.466147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a64a3-fd8a-499c-9e1b-7bf70ff7dd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:59.466414 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a64a3-fd8a-499c-9e1b-7bf70ff7dd6c admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:59.467158 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a64a3-fd8a-499c-9e1b-7bf70ff7dd6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:05:59.467474 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a64a3-fd8a-499c-9e1b-7bf70ff7dd6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:59.469156 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a64a3-fd8a-499c-9e1b-7bf70ff7dd6c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:59.484442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1156/2314] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:05:59.488561 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a64a3-fd8a-499c-9e1b-7bf70ff7dd6c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:59.488946 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae875cf0-42ac-452f-84d4-dc66f42f88fb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:59.502038 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a64a3-fd8a-499c-9e1b-7bf70ff7dd6c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:59.502962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b695b28-9458-4bcd-856a-4e369338304d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:59.503834 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b695b28-9458-4bcd-856a-4e369338304d admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:59.504820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b695b28-9458-4bcd-856a-4e369338304d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:59.504820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b695b28-9458-4bcd-856a-4e369338304d admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:59.505975 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b695b28-9458-4bcd-856a-4e369338304d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:59.508040 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b695b28-9458-4bcd-856a-4e369338304d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:59.521608 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab0ec0-daa2-4fd9-ac4b-0db8e8774c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:59.524816 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab0ec0-daa2-4fd9-ac4b-0db8e8774c72 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:59.525113 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab0ec0-daa2-4fd9-ac4b-0db8e8774c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:59.525417 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab0ec0-daa2-4fd9-ac4b-0db8e8774c72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:59.529229 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bab0ec0-daa2-4fd9-ac4b-0db8e8774c72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:59.532731 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bab0ec0-daa2-4fd9-ac4b-0db8e8774c72 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:59.560239 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1159/2315] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:59 2022] POST /identity/v3/projects => generated 383 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:05:59.561141 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b695b28-9458-4bcd-856a-4e369338304d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:59.588925 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1157/2316] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:05:59 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:59.594578 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab0ec0-daa2-4fd9-ac4b-0db8e8774c72 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:59.678035 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:59.700629 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:05:59.707020 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b695b28-9458-4bcd-856a-4e369338304d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:59.708821 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab0ec0-daa2-4fd9-ac4b-0db8e8774c72 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:59.714141 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6abe657-1f40-4a9c-9dcd-c3eb9fbc13fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:59.715563 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6abe657-1f40-4a9c-9dcd-c3eb9fbc13fb admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:59.715860 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6abe657-1f40-4a9c-9dcd-c3eb9fbc13fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:59.716145 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6abe657-1f40-4a9c-9dcd-c3eb9fbc13fb admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:59.716767 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c111cf-eb9b-46d4-ac89-c66488611acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:59.716899 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6abe657-1f40-4a9c-9dcd-c3eb9fbc13fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:59.717794 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c111cf-eb9b-46d4-ac89-c66488611acb admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:59.718289 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c111cf-eb9b-46d4-ac89-c66488611acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:59.718752 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c111cf-eb9b-46d4-ac89-c66488611acb admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:59.718953 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6abe657-1f40-4a9c-9dcd-c3eb9fbc13fb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:59.719815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c111cf-eb9b-46d4-ac89-c66488611acb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:59.722690 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c111cf-eb9b-46d4-ac89-c66488611acb admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:59.754645 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1160/2317] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:59 2022] POST /identity/v3/users => generated 298 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:59.756254 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6abe657-1f40-4a9c-9dcd-c3eb9fbc13fb admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:59.761183 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1158/2318] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:05:59 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:05:59.762927 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c111cf-eb9b-46d4-ac89-c66488611acb admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:59.778475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6abe657-1f40-4a9c-9dcd-c3eb9fbc13fb admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:59.782970 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c111cf-eb9b-46d4-ac89-c66488611acb admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:59.786231 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e94296-b101-4640-bbdb-d4e2626885b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:59.786968 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e94296-b101-4640-bbdb-d4e2626885b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:59.787262 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e94296-b101-4640-bbdb-d4e2626885b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:59.787544 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e94296-b101-4640-bbdb-d4e2626885b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:59.788350 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e94296-b101-4640-bbdb-d4e2626885b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:59.790042 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e94296-b101-4640-bbdb-d4e2626885b8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:59.795647 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4397b947-e926-4c63-af49-a976ae33c00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:59.796340 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397b947-e926-4c63-af49-a976ae33c00f admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:59.796626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397b947-e926-4c63-af49-a976ae33c00f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:59.796914 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397b947-e926-4c63-af49-a976ae33c00f admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:59.801698 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4397b947-e926-4c63-af49-a976ae33c00f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:59.803885 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4397b947-e926-4c63-af49-a976ae33c00f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:59.807432 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1161/2319] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:59.808373 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e94296-b101-4640-bbdb-d4e2626885b8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:59.821280 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e94296-b101-4640-bbdb-d4e2626885b8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:59.827206 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b21b9e-94c4-41fb-b13c-f5f6f3db5d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:59.828101 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b21b9e-94c4-41fb-b13c-f5f6f3db5d31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:59.828397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b21b9e-94c4-41fb-b13c-f5f6f3db5d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:59.828684 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b21b9e-94c4-41fb-b13c-f5f6f3db5d31 admin admin] PATH_INFO: `/v3/projects/b5780c0ecf764b40a86fc6c836ae64c9/users/f21556c4c53e4e99b824269370f42a61/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:59.840308 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1159/2320] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:05:59.844357 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4397b947-e926-4c63-af49-a976ae33c00f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:59.854486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b21b9e-94c4-41fb-b13c-f5f6f3db5d31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5780c0ecf764b40a86fc6c836ae64c9, user_id=f21556c4c53e4e99b824269370f42a61, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:59.857359 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b21b9e-94c4-41fb-b13c-f5f6f3db5d31 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:59.862327 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4397b947-e926-4c63-af49-a976ae33c00f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:59.870208 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47655f47-de4c-4b26-ab34-ab9cc02477f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:59.870816 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47655f47-de4c-4b26-ab34-ab9cc02477f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:59.874975 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47655f47-de4c-4b26-ab34-ab9cc02477f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:59.874975 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47655f47-de4c-4b26-ab34-ab9cc02477f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:59.874975 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47655f47-de4c-4b26-ab34-ab9cc02477f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:59.878606 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47655f47-de4c-4b26-ab34-ab9cc02477f8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:59.884262 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1162/2321] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:59 2022] PUT /identity/v3/projects/b5780c0ecf764b40a86fc6c836ae64c9/users/f21556c4c53e4e99b824269370f42a61/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:05:59.885296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b21b9e-94c4-41fb-b13c-f5f6f3db5d31 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:59.893992 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1160/2322] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 16:05:59.898765 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47655f47-de4c-4b26-ab34-ab9cc02477f8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:59.910411 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47655f47-de4c-4b26-ab34-ab9cc02477f8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:59.971140 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b21b9e-94c4-41fb-b13c-f5f6f3db5d31 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:05:59.976918 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c26c0-a3b5-4551-92ad-66eba555950e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:59.977790 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c26c0-a3b5-4551-92ad-66eba555950e admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:59.978501 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c26c0-a3b5-4551-92ad-66eba555950e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:59.978501 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c26c0-a3b5-4551-92ad-66eba555950e admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:05:59.979248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c26c0-a3b5-4551-92ad-66eba555950e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:05:59.981275 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c26c0-a3b5-4551-92ad-66eba555950e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:05:59.993627 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1163/2323] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 16:05:59.996827 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c26c0-a3b5-4551-92ad-66eba555950e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:05:59.997380 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf37f8-dcdb-4d73-8cd8-ab98b9489b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:05:59.998457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf37f8-dcdb-4d73-8cd8-ab98b9489b02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:05:59.998738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf37f8-dcdb-4d73-8cd8-ab98b9489b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:05:59.999019 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf37f8-dcdb-4d73-8cd8-ab98b9489b02 admin admin] PATH_INFO: `/v3/projects/a0fd322ae2f742ec8d94ab0efdf8f3f2/users/f544896008f548f28d0c0863c662b842/roles/54195e660fc349978b39d5794765a095` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:00.007738 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c26c0-a3b5-4551-92ad-66eba555950e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:00.013483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ef8b3-8900-4e13-b6c8-fd33ea24e7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:00.014080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ef8b3-8900-4e13-b6c8-fd33ea24e7a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:00.014374 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ef8b3-8900-4e13-b6c8-fd33ea24e7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:00.014676 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ef8b3-8900-4e13-b6c8-fd33ea24e7a2 admin admin] PATH_INFO: `/v3/projects/b5780c0ecf764b40a86fc6c836ae64c9/users/f21556c4c53e4e99b824269370f42a61/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:00.017461 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0ef8b3-8900-4e13-b6c8-fd33ea24e7a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5780c0ecf764b40a86fc6c836ae64c9, user_id=f21556c4c53e4e99b824269370f42a61, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:00.019084 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf37f8-dcdb-4d73-8cd8-ab98b9489b02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0fd322ae2f742ec8d94ab0efdf8f3f2, user_id=f544896008f548f28d0c0863c662b842, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:00.019785 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0ef8b3-8900-4e13-b6c8-fd33ea24e7a2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:00.022556 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf37f8-dcdb-4d73-8cd8-ab98b9489b02 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:00.040632 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1164/2324] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:59 2022] PUT /identity/v3/projects/b5780c0ecf764b40a86fc6c836ae64c9/users/f21556c4c53e4e99b824269370f42a61/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:06:00.043917 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a92ba-873d-4c99-8350-3f8eba6c24d9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:00.044334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a92ba-873d-4c99-8350-3f8eba6c24d9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:00.044810 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a92ba-873d-4c99-8350-3f8eba6c24d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:00.047213 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1161/2325] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:05:59 2022] PUT /identity/v3/projects/a0fd322ae2f742ec8d94ab0efdf8f3f2/users/f544896008f548f28d0c0863c662b842/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:06:00.067195 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978b314-d67d-442d-9334-1c8fa35c70a8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:00.067488 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978b314-d67d-442d-9334-1c8fa35c70a8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:00.068294 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978b314-d67d-442d-9334-1c8fa35c70a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:00.083222 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-708a92ba-873d-4c99-8350-3f8eba6c24d9 None None] MFA Rules not processed for user `f21556c4c53e4e99b824269370f42a61`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:06:00.085076 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-708a92ba-873d-4c99-8350-3f8eba6c24d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:00.095887 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2978b314-d67d-442d-9334-1c8fa35c70a8 None None] MFA Rules not processed for user `f544896008f548f28d0c0863c662b842`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:06:00.097137 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2978b314-d67d-442d-9334-1c8fa35c70a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:00.157359 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1165/2326] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:06:00 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:06:00.162981 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1162/2327] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:06:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:06:00.164949 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a92ba-873d-4c99-8350-3f8eba6c24d9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:00.166426 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-708a92ba-873d-4c99-8350-3f8eba6c24d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:00.174298 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2978b314-d67d-442d-9334-1c8fa35c70a8 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:00.175684 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2978b314-d67d-442d-9334-1c8fa35c70a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:00.188108 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:06:00.204118 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:06:00.226991 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a92ba-873d-4c99-8350-3f8eba6c24d9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:00.230612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09b04e-490d-408e-aeec-8e776cdd05e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:00.231257 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09b04e-490d-408e-aeec-8e776cdd05e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:00.231538 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09b04e-490d-408e-aeec-8e776cdd05e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:00.231841 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09b04e-490d-408e-aeec-8e776cdd05e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:00.236885 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2978b314-d67d-442d-9334-1c8fa35c70a8 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:00.243907 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4a09b04e-490d-408e-aeec-8e776cdd05e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:00.246347 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc59e0d7-1f51-4a7a-b893-8f0374257686 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:00.247007 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59e0d7-1f51-4a7a-b893-8f0374257686 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:00.247297 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59e0d7-1f51-4a7a-b893-8f0374257686 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:00.247577 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59e0d7-1f51-4a7a-b893-8f0374257686 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:00.257211 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc59e0d7-1f51-4a7a-b893-8f0374257686 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:00.258716 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc59e0d7-1f51-4a7a-b893-8f0374257686 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:00.268014 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1163/2328] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:06:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 16:06:00.302983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a09b04e-490d-408e-aeec-8e776cdd05e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:00.304435 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a09b04e-490d-408e-aeec-8e776cdd05e7 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:00.313922 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1166/2329] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:06:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 16:06:01.882192 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc59e0d7-1f51-4a7a-b893-8f0374257686 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:01.891470 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc59e0d7-1f51-4a7a-b893-8f0374257686 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:01.895227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95e378-624c-4d56-9be3-79b8958695db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:01.896409 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95e378-624c-4d56-9be3-79b8958695db service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:01.896695 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95e378-624c-4d56-9be3-79b8958695db service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:01.896977 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95e378-624c-4d56-9be3-79b8958695db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:01.954587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b95e378-624c-4d56-9be3-79b8958695db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:01.956025 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b95e378-624c-4d56-9be3-79b8958695db service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:01.965524 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1164/2330] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:06:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:06:03.334761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09b04e-490d-408e-aeec-8e776cdd05e7 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:03.343282 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09b04e-490d-408e-aeec-8e776cdd05e7 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:03.346890 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eaf5cc-d532-4ddc-b782-bb1dc6e077e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:03.347524 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eaf5cc-d532-4ddc-b782-bb1dc6e077e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:03.347812 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eaf5cc-d532-4ddc-b782-bb1dc6e077e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:03.348353 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eaf5cc-d532-4ddc-b782-bb1dc6e077e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:03.357221 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eaf5cc-d532-4ddc-b782-bb1dc6e077e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:03.358677 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eaf5cc-d532-4ddc-b782-bb1dc6e077e6 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:03.368613 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1167/2331] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:06:03 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:06:08.440034 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95e378-624c-4d56-9be3-79b8958695db service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:08.501349 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95e378-624c-4d56-9be3-79b8958695db service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:08.506317 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2cf74c-4e92-490d-bb40-265b56782a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:08.507179 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2cf74c-4e92-490d-bb40-265b56782a3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:08.507458 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2cf74c-4e92-490d-bb40-265b56782a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:08.507734 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2cf74c-4e92-490d-bb40-265b56782a3e admin admin] PATH_INFO: `/v3/users/f1e915375ff14eb1a54fef6a9cc8e4a0` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:08.508921 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2cf74c-4e92-490d-bb40-265b56782a3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1e915375ff14eb1a54fef6a9cc8e4a0)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:08.510873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2cf74c-4e92-490d-bb40-265b56782a3e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:08.586547 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cc2cf74c-4e92-490d-bb40-265b56782a3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1e915375ff14eb1a54fef6a9cc8e4a0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:08.587215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cc2cf74c-4e92-490d-bb40-265b56782a3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1e915375ff14eb1a54fef6a9cc8e4a0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:08.587830 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cc2cf74c-4e92-490d-bb40-265b56782a3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1e915375ff14eb1a54fef6a9cc8e4a0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:08.595853 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cc2cf74c-4e92-490d-bb40-265b56782a3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1e915375ff14eb1a54fef6a9cc8e4a0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:08.606468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cc2cf74c-4e92-490d-bb40-265b56782a3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1e915375ff14eb1a54fef6a9cc8e4a0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:08.613883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:06:08.633628 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1165/2332] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:06:08 2022] DELETE /identity/v3/users/f1e915375ff14eb1a54fef6a9cc8e4a0 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:06:08.640502 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3fe2e-26ed-4976-bfe3-bbe3abe74096 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:08.641448 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3fe2e-26ed-4976-bfe3-bbe3abe74096 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:08.641758 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3fe2e-26ed-4976-bfe3-bbe3abe74096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:08.660766 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4cd3fe2e-26ed-4976-bfe3-bbe3abe74096 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:06:08.662140 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4cd3fe2e-26ed-4976-bfe3-bbe3abe74096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:08.756073 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2cf74c-4e92-490d-bb40-265b56782a3e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:08.757457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cc2cf74c-4e92-490d-bb40-265b56782a3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:08.795020 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1168/2333] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:06:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:06:08.849128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd3fe2e-26ed-4976-bfe3-bbe3abe74096 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:08.850867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4cd3fe2e-26ed-4976-bfe3-bbe3abe74096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:08.885412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2cf74c-4e92-490d-bb40-265b56782a3e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:08.892063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31148e08-9944-444b-abc6-e932c1788743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:08.893042 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31148e08-9944-444b-abc6-e932c1788743 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:08.893512 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31148e08-9944-444b-abc6-e932c1788743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:08.894890 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31148e08-9944-444b-abc6-e932c1788743 admin admin] PATH_INFO: `/v3/users/f544896008f548f28d0c0863c662b842` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:08.897013 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31148e08-9944-444b-abc6-e932c1788743 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f544896008f548f28d0c0863c662b842)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:08.900267 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31148e08-9944-444b-abc6-e932c1788743 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:08.931775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd3fe2e-26ed-4976-bfe3-bbe3abe74096 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:08.936833 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e8cc8-5368-438b-a588-141e005bb94e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:08.937673 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e8cc8-5368-438b-a588-141e005bb94e service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:08.937988 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e8cc8-5368-438b-a588-141e005bb94e service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:08.938273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e8cc8-5368-438b-a588-141e005bb94e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:08.950797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e8cc8-5368-438b-a588-141e005bb94e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:08.953652 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e8cc8-5368-438b-a588-141e005bb94e service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:08.968859 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1169/2334] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:06:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:06:08.972188 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-31148e08-9944-444b-abc6-e932c1788743 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f544896008f548f28d0c0863c662b842'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:08.973155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-31148e08-9944-444b-abc6-e932c1788743 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f544896008f548f28d0c0863c662b842'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:08.974506 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-31148e08-9944-444b-abc6-e932c1788743 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f544896008f548f28d0c0863c662b842'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:08.982926 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-31148e08-9944-444b-abc6-e932c1788743 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f544896008f548f28d0c0863c662b842'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:08.992315 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-31148e08-9944-444b-abc6-e932c1788743 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f544896008f548f28d0c0863c662b842'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:09.015567 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1166/2335] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:06:08 2022] DELETE /identity/v3/users/f544896008f548f28d0c0863c662b842 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:06:09.020241 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c7556-53f9-42a3-9c8b-140b4abc887a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:09.020625 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c7556-53f9-42a3-9c8b-140b4abc887a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:09.020912 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c7556-53f9-42a3-9c8b-140b4abc887a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:09.029324 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:06:09.048147 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b11c7556-53f9-42a3-9c8b-140b4abc887a None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:06:09.049670 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b11c7556-53f9-42a3-9c8b-140b4abc887a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:09.217623 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1170/2336] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:06:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:06:09.229759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31148e08-9944-444b-abc6-e932c1788743 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:09.231311 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-31148e08-9944-444b-abc6-e932c1788743 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:09.312121 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31148e08-9944-444b-abc6-e932c1788743 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:09.325592 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f416022-4d14-4fdd-84b2-5e30e08dc703 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:09.326281 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f416022-4d14-4fdd-84b2-5e30e08dc703 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:09.326573 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f416022-4d14-4fdd-84b2-5e30e08dc703 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:09.326856 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f416022-4d14-4fdd-84b2-5e30e08dc703 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:09.339930 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f416022-4d14-4fdd-84b2-5e30e08dc703 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:09.341680 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f416022-4d14-4fdd-84b2-5e30e08dc703 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:09.354082 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1167/2337] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:06:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:06:09.512331 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c7556-53f9-42a3-9c8b-140b4abc887a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:09.515972 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b11c7556-53f9-42a3-9c8b-140b4abc887a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:09.527391 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c7556-53f9-42a3-9c8b-140b4abc887a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:09.533225 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5410ac-58d8-4876-8c9b-3d3858fc809c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:09.534408 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5410ac-58d8-4876-8c9b-3d3858fc809c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:09.534763 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5410ac-58d8-4876-8c9b-3d3858fc809c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:09.535593 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5410ac-58d8-4876-8c9b-3d3858fc809c admin admin] PATH_INFO: `/v3/projects/ee3300f56e7346ee8ecc93683e200c48` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:09.537784 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5410ac-58d8-4876-8c9b-3d3858fc809c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee3300f56e7346ee8ecc93683e200c48)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:09.542114 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5410ac-58d8-4876-8c9b-3d3858fc809c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:09.676589 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0c5410ac-58d8-4876-8c9b-3d3858fc809c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee3300f56e7346ee8ecc93683e200c48', 'request_id': 'req-0c5410ac-58d8-4876-8c9b-3d3858fc809c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:09.699096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0c5410ac-58d8-4876-8c9b-3d3858fc809c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee3300f56e7346ee8ecc93683e200c48', 'request_id': 'req-0c5410ac-58d8-4876-8c9b-3d3858fc809c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:09.728486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0c5410ac-58d8-4876-8c9b-3d3858fc809c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee3300f56e7346ee8ecc93683e200c48', 'request_id': 'req-0c5410ac-58d8-4876-8c9b-3d3858fc809c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:09.747164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0c5410ac-58d8-4876-8c9b-3d3858fc809c admin admin] The token cache is being invalidate because project ee3300f56e7346ee8ecc93683e200c48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:06:09.747446 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0c5410ac-58d8-4876-8c9b-3d3858fc809c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:09.748654 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0c5410ac-58d8-4876-8c9b-3d3858fc809c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:09.752163 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1171/2338] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:06:09 2022] DELETE /identity/v3/projects/ee3300f56e7346ee8ecc93683e200c48 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:06:09.757001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f416022-4d14-4fdd-84b2-5e30e08dc703 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:09.769889 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f416022-4d14-4fdd-84b2-5e30e08dc703 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:09.777104 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:06:09.839212 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec26dd2-c1f1-4cc6-8096-424163de8eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:09.840126 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec26dd2-c1f1-4cc6-8096-424163de8eab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:09.840438 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec26dd2-c1f1-4cc6-8096-424163de8eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:09.840747 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec26dd2-c1f1-4cc6-8096-424163de8eab admin admin] PATH_INFO: `/v3/users/bfceef4921f84a928c66f5990ed19388` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:09.842001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec26dd2-c1f1-4cc6-8096-424163de8eab admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfceef4921f84a928c66f5990ed19388)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:09.845617 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec26dd2-c1f1-4cc6-8096-424163de8eab admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:09.904870 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2ec26dd2-c1f1-4cc6-8096-424163de8eab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfceef4921f84a928c66f5990ed19388'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:09.904870 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2ec26dd2-c1f1-4cc6-8096-424163de8eab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfceef4921f84a928c66f5990ed19388'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:09.904870 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2ec26dd2-c1f1-4cc6-8096-424163de8eab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfceef4921f84a928c66f5990ed19388'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:09.911655 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2ec26dd2-c1f1-4cc6-8096-424163de8eab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfceef4921f84a928c66f5990ed19388'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:09.919541 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2ec26dd2-c1f1-4cc6-8096-424163de8eab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfceef4921f84a928c66f5990ed19388'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:09.941102 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1168/2339] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:06:09 2022] DELETE /identity/v3/users/bfceef4921f84a928c66f5990ed19388 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:06:09.996352 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5410ac-58d8-4876-8c9b-3d3858fc809c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:09.997894 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0c5410ac-58d8-4876-8c9b-3d3858fc809c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:10.011937 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5410ac-58d8-4876-8c9b-3d3858fc809c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:10.075721 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4992ad0-0319-41f9-8fb9-525b4b6322db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:10.076589 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4992ad0-0319-41f9-8fb9-525b4b6322db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:10.076881 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4992ad0-0319-41f9-8fb9-525b4b6322db admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:10.077161 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4992ad0-0319-41f9-8fb9-525b4b6322db admin admin] PATH_INFO: `/v3/projects/a0fd322ae2f742ec8d94ab0efdf8f3f2` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:10.078384 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4992ad0-0319-41f9-8fb9-525b4b6322db admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0fd322ae2f742ec8d94ab0efdf8f3f2)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:10.080306 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4992ad0-0319-41f9-8fb9-525b4b6322db admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:10.117172 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:06:10.133018 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a4992ad0-0319-41f9-8fb9-525b4b6322db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0fd322ae2f742ec8d94ab0efdf8f3f2', 'request_id': 'req-a4992ad0-0319-41f9-8fb9-525b4b6322db'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:10.140588 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a4992ad0-0319-41f9-8fb9-525b4b6322db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0fd322ae2f742ec8d94ab0efdf8f3f2', 'request_id': 'req-a4992ad0-0319-41f9-8fb9-525b4b6322db'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:10.146969 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a4992ad0-0319-41f9-8fb9-525b4b6322db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0fd322ae2f742ec8d94ab0efdf8f3f2', 'request_id': 'req-a4992ad0-0319-41f9-8fb9-525b4b6322db'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:10.158095 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a4992ad0-0319-41f9-8fb9-525b4b6322db admin admin] The token cache is being invalidate because project a0fd322ae2f742ec8d94ab0efdf8f3f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:06:10.158388 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a4992ad0-0319-41f9-8fb9-525b4b6322db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:10.159041 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a4992ad0-0319-41f9-8fb9-525b4b6322db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:10.160584 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1172/2340] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:06:09 2022] DELETE /identity/v3/projects/a0fd322ae2f742ec8d94ab0efdf8f3f2 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:06:10.165677 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f9198-b98f-4ac8-b344-155434246086 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:10.165977 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f9198-b98f-4ac8-b344-155434246086 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:10.166263 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f9198-b98f-4ac8-b344-155434246086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:10.186302 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0e6f9198-b98f-4ac8-b344-155434246086 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:06:10.187591 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0e6f9198-b98f-4ac8-b344-155434246086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:10.265778 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4992ad0-0319-41f9-8fb9-525b4b6322db admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:10.268698 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a4992ad0-0319-41f9-8fb9-525b4b6322db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:10.272977 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1169/2341] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:06:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:06:10.281523 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb196a2-2a42-46a6-bbda-5b65cbc17ed2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:10.282035 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb196a2-2a42-46a6-bbda-5b65cbc17ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:10.282417 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb196a2-2a42-46a6-bbda-5b65cbc17ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:10.307547 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-dfb196a2-2a42-46a6-bbda-5b65cbc17ed2 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:06:10.308775 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-dfb196a2-2a42-46a6-bbda-5b65cbc17ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:10.311922 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4992ad0-0319-41f9-8fb9-525b4b6322db admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:10.319258 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1170/2342] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:06:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:06:10.319996 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f3160-f98e-4a56-b0ea-2783487d5454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:10.320975 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f3160-f98e-4a56-b0ea-2783487d5454 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:10.321335 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f3160-f98e-4a56-b0ea-2783487d5454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:10.321740 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f3160-f98e-4a56-b0ea-2783487d5454 admin admin] PATH_INFO: `/v3/projects/a8668dd5a9504d37ba3748324ade9beb` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:10.323039 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f3160-f98e-4a56-b0ea-2783487d5454 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8668dd5a9504d37ba3748324ade9beb)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:10.323828 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb196a2-2a42-46a6-bbda-5b65cbc17ed2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:10.327207 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f3160-f98e-4a56-b0ea-2783487d5454 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:10.338099 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb196a2-2a42-46a6-bbda-5b65cbc17ed2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:10.343519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe80b5-0f87-49d0-a5f3-e7eb2c0f02bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:10.344418 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe80b5-0f87-49d0-a5f3-e7eb2c0f02bb admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:10.345221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe80b5-0f87-49d0-a5f3-e7eb2c0f02bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:10.345221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe80b5-0f87-49d0-a5f3-e7eb2c0f02bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:10.349855 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe80b5-0f87-49d0-a5f3-e7eb2c0f02bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:06:10.350455 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe80b5-0f87-49d0-a5f3-e7eb2c0f02bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:10.352673 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe80b5-0f87-49d0-a5f3-e7eb2c0f02bb admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:10.371373 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1171/2343] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:06:10.376190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe80b5-0f87-49d0-a5f3-e7eb2c0f02bb admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:10.382943 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-174f3160-f98e-4a56-b0ea-2783487d5454 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8668dd5a9504d37ba3748324ade9beb', 'request_id': 'req-174f3160-f98e-4a56-b0ea-2783487d5454'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:10.388766 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe80b5-0f87-49d0-a5f3-e7eb2c0f02bb admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:10.392962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-174f3160-f98e-4a56-b0ea-2783487d5454 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8668dd5a9504d37ba3748324ade9beb', 'request_id': 'req-174f3160-f98e-4a56-b0ea-2783487d5454'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:10.394180 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:06:10.402593 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-174f3160-f98e-4a56-b0ea-2783487d5454 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8668dd5a9504d37ba3748324ade9beb', 'request_id': 'req-174f3160-f98e-4a56-b0ea-2783487d5454'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:10.420101 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-174f3160-f98e-4a56-b0ea-2783487d5454 admin admin] The token cache is being invalidate because project a8668dd5a9504d37ba3748324ade9beb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:06:10.420477 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-174f3160-f98e-4a56-b0ea-2783487d5454 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:10.421209 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-174f3160-f98e-4a56-b0ea-2783487d5454 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:10.427453 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1173/2344] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:06:10 2022] DELETE /identity/v3/projects/a8668dd5a9504d37ba3748324ade9beb => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:06:10.428440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f3160-f98e-4a56-b0ea-2783487d5454 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:10.432750 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-174f3160-f98e-4a56-b0ea-2783487d5454 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:10.467771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff79ed-3481-4a6c-bcc0-78b1934ede9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:10.468434 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff79ed-3481-4a6c-bcc0-78b1934ede9c admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:10.468724 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff79ed-3481-4a6c-bcc0-78b1934ede9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:10.469002 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff79ed-3481-4a6c-bcc0-78b1934ede9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:10.469786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ff79ed-3481-4a6c-bcc0-78b1934ede9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:10.471808 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ff79ed-3481-4a6c-bcc0-78b1934ede9c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:10.498503 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1172/2345] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:06:10 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:06:10.502290 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f3160-f98e-4a56-b0ea-2783487d5454 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:10.502729 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff79ed-3481-4a6c-bcc0-78b1934ede9c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:10.504431 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-25ff79ed-3481-4a6c-bcc0-78b1934ede9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:10.547605 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:06:10.562228 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d25e933-eeda-4b58-87f4-503c8de3fa61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:10.563188 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25e933-eeda-4b58-87f4-503c8de3fa61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:10.563542 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25e933-eeda-4b58-87f4-503c8de3fa61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:10.563882 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25e933-eeda-4b58-87f4-503c8de3fa61 admin admin] PATH_INFO: `/v3/users/f21556c4c53e4e99b824269370f42a61` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:10.565150 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d25e933-eeda-4b58-87f4-503c8de3fa61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f21556c4c53e4e99b824269370f42a61)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:10.567100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d25e933-eeda-4b58-87f4-503c8de3fa61 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:10.571324 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff79ed-3481-4a6c-bcc0-78b1934ede9c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:10.576350 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b946c7b-4ab2-437c-baae-7f3028cbfa8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:10.577086 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b946c7b-4ab2-437c-baae-7f3028cbfa8f admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:10.577438 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b946c7b-4ab2-437c-baae-7f3028cbfa8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:10.577812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b946c7b-4ab2-437c-baae-7f3028cbfa8f admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:10.578576 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b946c7b-4ab2-437c-baae-7f3028cbfa8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:10.580688 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b946c7b-4ab2-437c-baae-7f3028cbfa8f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:10.610601 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1173/2346] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:06:10 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:06:10.614554 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b946c7b-4ab2-437c-baae-7f3028cbfa8f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:10.625128 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b946c7b-4ab2-437c-baae-7f3028cbfa8f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:10.627282 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8d25e933-eeda-4b58-87f4-503c8de3fa61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f21556c4c53e4e99b824269370f42a61'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:10.627978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8d25e933-eeda-4b58-87f4-503c8de3fa61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f21556c4c53e4e99b824269370f42a61'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:10.628602 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8d25e933-eeda-4b58-87f4-503c8de3fa61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f21556c4c53e4e99b824269370f42a61'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:10.630934 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dd9069-fede-42e4-8e80-5deb95dc9877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:10.632165 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd9069-fede-42e4-8e80-5deb95dc9877 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:10.632555 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd9069-fede-42e4-8e80-5deb95dc9877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:10.632908 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd9069-fede-42e4-8e80-5deb95dc9877 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:10.633766 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dd9069-fede-42e4-8e80-5deb95dc9877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:10.635439 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dd9069-fede-42e4-8e80-5deb95dc9877 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:10.637167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8d25e933-eeda-4b58-87f4-503c8de3fa61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f21556c4c53e4e99b824269370f42a61'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:10.646889 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8d25e933-eeda-4b58-87f4-503c8de3fa61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f21556c4c53e4e99b824269370f42a61'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:10.647311 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1174/2347] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:06:10.651644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dd9069-fede-42e4-8e80-5deb95dc9877 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:10.653418 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e8dd9069-fede-42e4-8e80-5deb95dc9877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:10.669035 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dd9069-fede-42e4-8e80-5deb95dc9877 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:10.670653 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1174/2348] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:06:10 2022] DELETE /identity/v3/users/f21556c4c53e4e99b824269370f42a61 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:06:10.759618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a8afd-b33f-40c8-8589-a1b2eb13a234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:10.760958 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a8afd-b33f-40c8-8589-a1b2eb13a234 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:10.761260 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a8afd-b33f-40c8-8589-a1b2eb13a234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:10.761545 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a8afd-b33f-40c8-8589-a1b2eb13a234 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:10.762843 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416a8afd-b33f-40c8-8589-a1b2eb13a234 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:10.764853 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416a8afd-b33f-40c8-8589-a1b2eb13a234 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:10.779660 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1175/2349] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 16:06:10.784039 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d25e933-eeda-4b58-87f4-503c8de3fa61 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:10.803520 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d25e933-eeda-4b58-87f4-503c8de3fa61 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:10.825393 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d941438-a10b-4bc4-ba45-a7cc3e87cd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:10.826992 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d941438-a10b-4bc4-ba45-a7cc3e87cd6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:10.827752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d941438-a10b-4bc4-ba45-a7cc3e87cd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:10.828117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d941438-a10b-4bc4-ba45-a7cc3e87cd6c admin admin] PATH_INFO: `/v3/projects/a74296f7e4d0463d9cb73f0c9162ccba/users/7c9f40cb09a34ce2a9ff51da0d0e5954/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:10.857091 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d941438-a10b-4bc4-ba45-a7cc3e87cd6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a74296f7e4d0463d9cb73f0c9162ccba, user_id=7c9f40cb09a34ce2a9ff51da0d0e5954, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:10.874322 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d941438-a10b-4bc4-ba45-a7cc3e87cd6c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:10.907475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1175/2350] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:06:10 2022] PUT /identity/v3/projects/a74296f7e4d0463d9cb73f0c9162ccba/users/7c9f40cb09a34ce2a9ff51da0d0e5954/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:06:10.912510 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ded684-7f5e-47e9-be69-998bfd6e2507 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:10.913239 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ded684-7f5e-47e9-be69-998bfd6e2507 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:10.913390 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ded684-7f5e-47e9-be69-998bfd6e2507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:10.950106 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:06:10.975838 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b3ded684-7f5e-47e9-be69-998bfd6e2507 None None] MFA Rules not processed for user `7c9f40cb09a34ce2a9ff51da0d0e5954`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:06:10.977288 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b3ded684-7f5e-47e9-be69-998bfd6e2507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:11.080580 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1176/2351] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:06:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:06:11.086224 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d941438-a10b-4bc4-ba45-a7cc3e87cd6c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:11.107898 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ded684-7f5e-47e9-be69-998bfd6e2507 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:11.109616 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b3ded684-7f5e-47e9-be69-998bfd6e2507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:11.173135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d941438-a10b-4bc4-ba45-a7cc3e87cd6c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:11.178663 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9d156-8a94-4259-9b1b-aea41b75c52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:11.179327 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9d156-8a94-4259-9b1b-aea41b75c52a admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:11.179612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9d156-8a94-4259-9b1b-aea41b75c52a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:11.179921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9d156-8a94-4259-9b1b-aea41b75c52a admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:11.180629 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d9d156-8a94-4259-9b1b-aea41b75c52a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:11.182760 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d9d156-8a94-4259-9b1b-aea41b75c52a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:11.209861 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ded684-7f5e-47e9-be69-998bfd6e2507 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:11.215086 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402a3dd-a780-4007-911c-e8515301afdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:11.215990 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1176/2352] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:06:11 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:06:11.216077 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402a3dd-a780-4007-911c-e8515301afdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:11.216357 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402a3dd-a780-4007-911c-e8515301afdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:11.216708 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402a3dd-a780-4007-911c-e8515301afdd admin admin] PATH_INFO: `/v3/projects/b5780c0ecf764b40a86fc6c836ae64c9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:11.216809 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9d156-8a94-4259-9b1b-aea41b75c52a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:11.217980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402a3dd-a780-4007-911c-e8515301afdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5780c0ecf764b40a86fc6c836ae64c9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:11.220502 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402a3dd-a780-4007-911c-e8515301afdd admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:11.252680 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:06:11.282299 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3402a3dd-a780-4007-911c-e8515301afdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5780c0ecf764b40a86fc6c836ae64c9', 'request_id': 'req-3402a3dd-a780-4007-911c-e8515301afdd'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:11.289421 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3402a3dd-a780-4007-911c-e8515301afdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5780c0ecf764b40a86fc6c836ae64c9', 'request_id': 'req-3402a3dd-a780-4007-911c-e8515301afdd'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:11.299664 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3402a3dd-a780-4007-911c-e8515301afdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5780c0ecf764b40a86fc6c836ae64c9', 'request_id': 'req-3402a3dd-a780-4007-911c-e8515301afdd'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:11.303894 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9d156-8a94-4259-9b1b-aea41b75c52a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:11.311288 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3402a3dd-a780-4007-911c-e8515301afdd admin admin] The token cache is being invalidate because project b5780c0ecf764b40a86fc6c836ae64c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:06:11.311550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3402a3dd-a780-4007-911c-e8515301afdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:11.312184 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3402a3dd-a780-4007-911c-e8515301afdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:11.316667 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1177/2353] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:06:11 2022] DELETE /identity/v3/projects/b5780c0ecf764b40a86fc6c836ae64c9 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:06:11.386017 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f697fef-ae97-4151-9524-2af9667d67a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:11.387331 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f697fef-ae97-4151-9524-2af9667d67a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:11.387768 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f697fef-ae97-4151-9524-2af9667d67a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:11.388173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f697fef-ae97-4151-9524-2af9667d67a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:11.389797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f697fef-ae97-4151-9524-2af9667d67a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:11.391998 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f697fef-ae97-4151-9524-2af9667d67a2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:11.418094 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1177/2354] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:06:11 2022] POST /identity/v3/users => generated 294 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:06:11.421784 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402a3dd-a780-4007-911c-e8515301afdd admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:11.423310 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3402a3dd-a780-4007-911c-e8515301afdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:11.442144 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402a3dd-a780-4007-911c-e8515301afdd admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:11.448008 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dca1fb-39f9-4970-b825-f0fd786f8369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:11.448822 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dca1fb-39f9-4970-b825-f0fd786f8369 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:11.449175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dca1fb-39f9-4970-b825-f0fd786f8369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:11.449520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dca1fb-39f9-4970-b825-f0fd786f8369 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:11.450377 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dca1fb-39f9-4970-b825-f0fd786f8369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:11.452102 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dca1fb-39f9-4970-b825-f0fd786f8369 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:11.465627 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1178/2355] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 16:06:11.469776 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f697fef-ae97-4151-9524-2af9667d67a2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:11.484063 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f697fef-ae97-4151-9524-2af9667d67a2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:11.489512 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163af2f7-b4b1-4342-a1f5-2555675b4b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:11.490539 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163af2f7-b4b1-4342-a1f5-2555675b4b6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:11.490888 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163af2f7-b4b1-4342-a1f5-2555675b4b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:11.491263 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163af2f7-b4b1-4342-a1f5-2555675b4b6b admin admin] PATH_INFO: `/v3/projects/fe65136df6c74761bbff14ddd920d0e1/users/37fec2e80ad54c64b2760c0433d1b022/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:11.512931 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163af2f7-b4b1-4342-a1f5-2555675b4b6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe65136df6c74761bbff14ddd920d0e1, user_id=37fec2e80ad54c64b2760c0433d1b022, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:11.515908 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163af2f7-b4b1-4342-a1f5-2555675b4b6b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:11.535152 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1178/2356] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:06:11 2022] PUT /identity/v3/projects/fe65136df6c74761bbff14ddd920d0e1/users/37fec2e80ad54c64b2760c0433d1b022/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:06:11.538979 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dca1fb-39f9-4970-b825-f0fd786f8369 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:11.552705 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dca1fb-39f9-4970-b825-f0fd786f8369 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:11.571632 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:06:11.648031 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c864d-eedd-4e63-99fd-b396235ff43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:11.648726 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c864d-eedd-4e63-99fd-b396235ff43c admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:11.649010 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c864d-eedd-4e63-99fd-b396235ff43c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:11.649287 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c864d-eedd-4e63-99fd-b396235ff43c admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:11.650069 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c864d-eedd-4e63-99fd-b396235ff43c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:11.652417 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c864d-eedd-4e63-99fd-b396235ff43c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:11.671783 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1179/2357] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:06:11.672937 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163af2f7-b4b1-4342-a1f5-2555675b4b6b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:11.683567 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163af2f7-b4b1-4342-a1f5-2555675b4b6b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:11.688179 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3769dc-85b6-4e43-9ec9-8a4fe7fc31da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:11.689057 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3769dc-85b6-4e43-9ec9-8a4fe7fc31da admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:11.689338 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3769dc-85b6-4e43-9ec9-8a4fe7fc31da admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:11.689639 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3769dc-85b6-4e43-9ec9-8a4fe7fc31da admin admin] PATH_INFO: `/v3/projects/fe65136df6c74761bbff14ddd920d0e1/users/37fec2e80ad54c64b2760c0433d1b022/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:11.691634 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3769dc-85b6-4e43-9ec9-8a4fe7fc31da admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe65136df6c74761bbff14ddd920d0e1, user_id=37fec2e80ad54c64b2760c0433d1b022, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:11.694723 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3769dc-85b6-4e43-9ec9-8a4fe7fc31da admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:11.704713 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1179/2358] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:06:11 2022] PUT /identity/v3/projects/fe65136df6c74761bbff14ddd920d0e1/users/37fec2e80ad54c64b2760c0433d1b022/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:06:11.715322 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2cc27-d039-4914-ad37-598661b24b89 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:11.715613 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2cc27-d039-4914-ad37-598661b24b89 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:11.715892 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2cc27-d039-4914-ad37-598661b24b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:11.743726 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-dfe2cc27-d039-4914-ad37-598661b24b89 None None] MFA Rules not processed for user `37fec2e80ad54c64b2760c0433d1b022`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:06:11.744937 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-dfe2cc27-d039-4914-ad37-598661b24b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:11.811703 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1180/2359] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:06:11 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:06:11.822494 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3769dc-85b6-4e43-9ec9-8a4fe7fc31da admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:11.824132 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-df3769dc-85b6-4e43-9ec9-8a4fe7fc31da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:11.880879 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3769dc-85b6-4e43-9ec9-8a4fe7fc31da admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:11.884712 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ada32-d577-4f90-9945-39523d397a33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:11.885339 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ada32-d577-4f90-9945-39523d397a33 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:11.885644 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ada32-d577-4f90-9945-39523d397a33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:11.885920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ada32-d577-4f90-9945-39523d397a33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:11.887652 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ca6ada32-d577-4f90-9945-39523d397a33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:11.930210 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:06:11.947864 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6ada32-d577-4f90-9945-39523d397a33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:11.949360 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6ada32-d577-4f90-9945-39523d397a33 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:11.958827 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1180/2360] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:06:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:06:15.878608 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758eb69d-32f9-4c2b-8763-ba33bf072857 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:15.879081 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758eb69d-32f9-4c2b-8763-ba33bf072857 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:15.879236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758eb69d-32f9-4c2b-8763-ba33bf072857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:15.898725 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-758eb69d-32f9-4c2b-8763-ba33bf072857 None None] MFA Rules not processed for user `6edbad98ed0f4823b9955f9f0e270149`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:06:15.899908 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-758eb69d-32f9-4c2b-8763-ba33bf072857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:15.959893 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1181/2361] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 21 16:06:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:06:16.011692 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ada32-d577-4f90-9945-39523d397a33 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:16.013109 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ca6ada32-d577-4f90-9945-39523d397a33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:16.085500 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ada32-d577-4f90-9945-39523d397a33 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:16.091838 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1db82-98cd-4a7b-b81b-643eedbe4f98 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5bfae2cb30843b1b1a0343efef38638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:16.092491 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1db82-98cd-4a7b-b81b-643eedbe4f98 service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:16.092776 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1db82-98cd-4a7b-b81b-643eedbe4f98 service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:16.093054 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1db82-98cd-4a7b-b81b-643eedbe4f98 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:16.102669 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca1db82-98cd-4a7b-b81b-643eedbe4f98 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:16.104307 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca1db82-98cd-4a7b-b81b-643eedbe4f98 service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:16.112497 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1181/2362] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 16:06:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 16:06:16.612822 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758eb69d-32f9-4c2b-8763-ba33bf072857 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:16.678099 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:06:16.704526 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758eb69d-32f9-4c2b-8763-ba33bf072857 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:16.710656 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c37670-b585-4e86-8d73-9e840e8adeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:16.711864 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c37670-b585-4e86-8d73-9e840e8adeb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:16.712155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c37670-b585-4e86-8d73-9e840e8adeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:16.712498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c37670-b585-4e86-8d73-9e840e8adeb6 admin admin] PATH_INFO: `/v3/users/7c9f40cb09a34ce2a9ff51da0d0e5954` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:16.714227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c37670-b585-4e86-8d73-9e840e8adeb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c9f40cb09a34ce2a9ff51da0d0e5954)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:16.716555 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c37670-b585-4e86-8d73-9e840e8adeb6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:16.796928 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-57c37670-b585-4e86-8d73-9e840e8adeb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c9f40cb09a34ce2a9ff51da0d0e5954'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:16.798709 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-57c37670-b585-4e86-8d73-9e840e8adeb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c9f40cb09a34ce2a9ff51da0d0e5954'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:16.800818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-57c37670-b585-4e86-8d73-9e840e8adeb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c9f40cb09a34ce2a9ff51da0d0e5954'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:16.818117 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-57c37670-b585-4e86-8d73-9e840e8adeb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c9f40cb09a34ce2a9ff51da0d0e5954'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:16.827532 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-57c37670-b585-4e86-8d73-9e840e8adeb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c9f40cb09a34ce2a9ff51da0d0e5954'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:16.853896 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1182/2363] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:06:16 2022] DELETE /identity/v3/users/7c9f40cb09a34ce2a9ff51da0d0e5954 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:06:16.858645 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02f0b9-eaed-42f1-bda7-9da3c8e6b306 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:16.858859 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02f0b9-eaed-42f1-bda7-9da3c8e6b306 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:16.859185 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02f0b9-eaed-42f1-bda7-9da3c8e6b306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:16.885565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1e02f0b9-eaed-42f1-bda7-9da3c8e6b306 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:06:16.887095 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1e02f0b9-eaed-42f1-bda7-9da3c8e6b306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:17.009101 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1182/2364] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:06:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:06:17.035385 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c37670-b585-4e86-8d73-9e840e8adeb6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:17.036882 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-57c37670-b585-4e86-8d73-9e840e8adeb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:17.126354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c37670-b585-4e86-8d73-9e840e8adeb6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:17.134362 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08df0d-8055-47f5-8a39-b1f017c893ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:17.135570 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08df0d-8055-47f5-8a39-b1f017c893ee service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:17.136066 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08df0d-8055-47f5-8a39-b1f017c893ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:17.136531 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08df0d-8055-47f5-8a39-b1f017c893ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:17.151193 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c08df0d-8055-47f5-8a39-b1f017c893ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:17.152991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c08df0d-8055-47f5-8a39-b1f017c893ee service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:17.166429 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1183/2365] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:06:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:06:17.543557 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02f0b9-eaed-42f1-bda7-9da3c8e6b306 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:17.545057 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1e02f0b9-eaed-42f1-bda7-9da3c8e6b306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:17.564402 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02f0b9-eaed-42f1-bda7-9da3c8e6b306 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:17.569630 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20498590-fc50-421b-8ae6-a12d306d11e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:17.570521 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20498590-fc50-421b-8ae6-a12d306d11e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:17.570837 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20498590-fc50-421b-8ae6-a12d306d11e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:17.571122 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20498590-fc50-421b-8ae6-a12d306d11e7 admin admin] PATH_INFO: `/v3/projects/a74296f7e4d0463d9cb73f0c9162ccba` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:17.572338 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20498590-fc50-421b-8ae6-a12d306d11e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a74296f7e4d0463d9cb73f0c9162ccba)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:17.574350 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20498590-fc50-421b-8ae6-a12d306d11e7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:17.667924 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-20498590-fc50-421b-8ae6-a12d306d11e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a74296f7e4d0463d9cb73f0c9162ccba', 'request_id': 'req-20498590-fc50-421b-8ae6-a12d306d11e7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:17.681882 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-20498590-fc50-421b-8ae6-a12d306d11e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a74296f7e4d0463d9cb73f0c9162ccba', 'request_id': 'req-20498590-fc50-421b-8ae6-a12d306d11e7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:17.690872 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-20498590-fc50-421b-8ae6-a12d306d11e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a74296f7e4d0463d9cb73f0c9162ccba', 'request_id': 'req-20498590-fc50-421b-8ae6-a12d306d11e7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:17.694321 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:06:17.706638 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-20498590-fc50-421b-8ae6-a12d306d11e7 admin admin] The token cache is being invalidate because project a74296f7e4d0463d9cb73f0c9162ccba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:06:17.707280 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-20498590-fc50-421b-8ae6-a12d306d11e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:17.708148 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-20498590-fc50-421b-8ae6-a12d306d11e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:17.710107 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1183/2366] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:06:17 2022] DELETE /identity/v3/projects/a74296f7e4d0463d9cb73f0c9162ccba => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:06:17.714133 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08df0d-8055-47f5-8a39-b1f017c893ee service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:17.726831 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08df0d-8055-47f5-8a39-b1f017c893ee service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:17.800812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3451511-d848-4199-bf73-515fc139d3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:17.802527 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3451511-d848-4199-bf73-515fc139d3e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:17.802924 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3451511-d848-4199-bf73-515fc139d3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:17.803300 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3451511-d848-4199-bf73-515fc139d3e3 admin admin] PATH_INFO: `/v3/users/37fec2e80ad54c64b2760c0433d1b022` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:17.804744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3451511-d848-4199-bf73-515fc139d3e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37fec2e80ad54c64b2760c0433d1b022)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:17.806863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3451511-d848-4199-bf73-515fc139d3e3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:17.812765 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:06:17.877250 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f3451511-d848-4199-bf73-515fc139d3e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37fec2e80ad54c64b2760c0433d1b022'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:17.878014 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f3451511-d848-4199-bf73-515fc139d3e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37fec2e80ad54c64b2760c0433d1b022'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:17.878652 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f3451511-d848-4199-bf73-515fc139d3e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37fec2e80ad54c64b2760c0433d1b022'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:17.894226 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f3451511-d848-4199-bf73-515fc139d3e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37fec2e80ad54c64b2760c0433d1b022'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:17.907749 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f3451511-d848-4199-bf73-515fc139d3e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37fec2e80ad54c64b2760c0433d1b022'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:17.934082 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1184/2367] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:06:17 2022] DELETE /identity/v3/users/37fec2e80ad54c64b2760c0433d1b022 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:06:18.338110 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20498590-fc50-421b-8ae6-a12d306d11e7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:18.340544 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-20498590-fc50-421b-8ae6-a12d306d11e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:18.365002 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9724ac6-e68d-4c46-9388-22ade9cec551 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:18.365464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9724ac6-e68d-4c46-9388-22ade9cec551 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:18.365607 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9724ac6-e68d-4c46-9388-22ade9cec551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:18.368464 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20498590-fc50-421b-8ae6-a12d306d11e7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:18.394201 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a9724ac6-e68d-4c46-9388-22ade9cec551 None None] MFA Rules not processed for user `a50cbf92286c420f9bee6e05c33dc440`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:06:18.396129 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a9724ac6-e68d-4c46-9388-22ade9cec551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:18.466369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346c8667-0c4f-495f-b67b-6db646a2b9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:18.468590 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c8667-0c4f-495f-b67b-6db646a2b9ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:18.469714 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c8667-0c4f-495f-b67b-6db646a2b9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:18.470233 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c8667-0c4f-495f-b67b-6db646a2b9ff admin admin] PATH_INFO: `/v3/projects/fe65136df6c74761bbff14ddd920d0e1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:18.472179 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346c8667-0c4f-495f-b67b-6db646a2b9ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe65136df6c74761bbff14ddd920d0e1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:18.474615 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346c8667-0c4f-495f-b67b-6db646a2b9ff admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:18.490553 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1185/2368] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:06:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:06:18.552748 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-346c8667-0c4f-495f-b67b-6db646a2b9ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe65136df6c74761bbff14ddd920d0e1', 'request_id': 'req-346c8667-0c4f-495f-b67b-6db646a2b9ff'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:18.571571 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-346c8667-0c4f-495f-b67b-6db646a2b9ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe65136df6c74761bbff14ddd920d0e1', 'request_id': 'req-346c8667-0c4f-495f-b67b-6db646a2b9ff'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:18.578469 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-346c8667-0c4f-495f-b67b-6db646a2b9ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe65136df6c74761bbff14ddd920d0e1', 'request_id': 'req-346c8667-0c4f-495f-b67b-6db646a2b9ff'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:18.602938 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-346c8667-0c4f-495f-b67b-6db646a2b9ff admin admin] The token cache is being invalidate because project fe65136df6c74761bbff14ddd920d0e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:06:18.603243 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-346c8667-0c4f-495f-b67b-6db646a2b9ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:18.603919 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-346c8667-0c4f-495f-b67b-6db646a2b9ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:18.609217 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1184/2369] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:06:18 2022] DELETE /identity/v3/projects/fe65136df6c74761bbff14ddd920d0e1 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:06:18.611512 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7dd60d-c575-4e8f-8a49-85889a41bb86 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:18.611512 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7dd60d-c575-4e8f-8a49-85889a41bb86 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:18.611512 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7dd60d-c575-4e8f-8a49-85889a41bb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:18.646915 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6c7dd60d-c575-4e8f-8a49-85889a41bb86 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:06:18.648163 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6c7dd60d-c575-4e8f-8a49-85889a41bb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:18.801358 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1186/2370] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:06:18.810487 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41acd44c-3a35-4473-aa63-465fa2889820 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:18.810840 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41acd44c-3a35-4473-aa63-465fa2889820 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:18.811046 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41acd44c-3a35-4473-aa63-465fa2889820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:18.876620 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-41acd44c-3a35-4473-aa63-465fa2889820 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:06:18.878843 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-41acd44c-3a35-4473-aa63-465fa2889820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:18.916702 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1185/2371] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:06:18.920041 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7dd60d-c575-4e8f-8a49-85889a41bb86 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:18.931925 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7dd60d-c575-4e8f-8a49-85889a41bb86 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:18.937715 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029528a0-4dc2-416f-a75b-90253e7111e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:18.938397 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029528a0-4dc2-416f-a75b-90253e7111e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:18.938682 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029528a0-4dc2-416f-a75b-90253e7111e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:18.938960 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029528a0-4dc2-416f-a75b-90253e7111e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:18.939797 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029528a0-4dc2-416f-a75b-90253e7111e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:06:18.940054 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029528a0-4dc2-416f-a75b-90253e7111e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:18.941730 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029528a0-4dc2-416f-a75b-90253e7111e0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:18.955328 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1187/2372] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:06:18.959992 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41acd44c-3a35-4473-aa63-465fa2889820 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:18.970454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41acd44c-3a35-4473-aa63-465fa2889820 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:18.979293 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d3862-feaf-4cd3-a8db-358521dfc9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:18.979976 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d3862-feaf-4cd3-a8db-358521dfc9fd admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:18.980266 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d3862-feaf-4cd3-a8db-358521dfc9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:18.980537 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d3862-feaf-4cd3-a8db-358521dfc9fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:18.981254 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d3862-feaf-4cd3-a8db-358521dfc9fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:18.983216 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d3862-feaf-4cd3-a8db-358521dfc9fd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:19.015680 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1186/2373] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:06:18 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:06:19.020844 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029528a0-4dc2-416f-a75b-90253e7111e0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:19.032392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029528a0-4dc2-416f-a75b-90253e7111e0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:19.055227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:06:19.120343 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d28ee-26f7-4df7-ad44-ebfde8f42957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:19.121727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d28ee-26f7-4df7-ad44-ebfde8f42957 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:19.122560 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d28ee-26f7-4df7-ad44-ebfde8f42957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:19.122941 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d28ee-26f7-4df7-ad44-ebfde8f42957 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:19.124265 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d28ee-26f7-4df7-ad44-ebfde8f42957 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:19.126903 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d28ee-26f7-4df7-ad44-ebfde8f42957 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:19.154969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1188/2374] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:06:19 2022] POST /identity/v3/users => generated 286 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:06:19.158770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d3862-feaf-4cd3-a8db-358521dfc9fd admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:19.171178 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d3862-feaf-4cd3-a8db-358521dfc9fd admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:19.185256 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4799c-affb-44c2-b2f9-8059f04370fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:19.186264 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4799c-affb-44c2-b2f9-8059f04370fa admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:19.186702 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4799c-affb-44c2-b2f9-8059f04370fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:19.186988 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4799c-affb-44c2-b2f9-8059f04370fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:19.187755 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a4799c-affb-44c2-b2f9-8059f04370fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:19.189527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a4799c-affb-44c2-b2f9-8059f04370fa admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:19.209337 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1187/2375] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 16:06:19.210777 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d28ee-26f7-4df7-ad44-ebfde8f42957 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:19.221625 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d28ee-26f7-4df7-ad44-ebfde8f42957 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:19.228408 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c3b56-41d2-49ea-ae40-f774b08b499b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:19.229312 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c3b56-41d2-49ea-ae40-f774b08b499b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:19.229756 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c3b56-41d2-49ea-ae40-f774b08b499b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:19.230153 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c3b56-41d2-49ea-ae40-f774b08b499b admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:19.231050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c3b56-41d2-49ea-ae40-f774b08b499b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:19.232770 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c3b56-41d2-49ea-ae40-f774b08b499b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:19.245429 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1189/2376] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:06:19.249490 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4799c-affb-44c2-b2f9-8059f04370fa admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:19.262746 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4799c-affb-44c2-b2f9-8059f04370fa admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:19.268597 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2686f90-d4ca-4ce8-a85b-58a4664613eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:19.269519 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2686f90-d4ca-4ce8-a85b-58a4664613eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:19.269834 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2686f90-d4ca-4ce8-a85b-58a4664613eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:19.270165 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2686f90-d4ca-4ce8-a85b-58a4664613eb admin admin] PATH_INFO: `/v3/projects/c74c7c3813da4410838333e3a8480093/users/009030f4b9394dd08d3f661269300f1d/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:19.288423 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2686f90-d4ca-4ce8-a85b-58a4664613eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c74c7c3813da4410838333e3a8480093, user_id=009030f4b9394dd08d3f661269300f1d, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:19.292833 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2686f90-d4ca-4ce8-a85b-58a4664613eb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:19.319241 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1188/2377] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:06:19 2022] PUT /identity/v3/projects/c74c7c3813da4410838333e3a8480093/users/009030f4b9394dd08d3f661269300f1d/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:06:19.321162 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0f494-fdcf-41ba-aeeb-708e07911b5a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:19.321376 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0f494-fdcf-41ba-aeeb-708e07911b5a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:19.321685 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0f494-fdcf-41ba-aeeb-708e07911b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:19.358789 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-21c0f494-fdcf-41ba-aeeb-708e07911b5a None None] MFA Rules not processed for user `009030f4b9394dd08d3f661269300f1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:06:19.360346 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-21c0f494-fdcf-41ba-aeeb-708e07911b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:19.509947 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1190/2378] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:06:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:06:19.518143 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2686f90-d4ca-4ce8-a85b-58a4664613eb admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:19.521528 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f2686f90-d4ca-4ce8-a85b-58a4664613eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:19.621203 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:06:19.728688 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2686f90-d4ca-4ce8-a85b-58a4664613eb admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:19.752381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400d8839-4a9c-487a-ad75-90bf69951aab service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73a7d5f37e144768d7cc53f13cd5157', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:19.753070 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d8839-4a9c-487a-ad75-90bf69951aab service glance] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:19.753366 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d8839-4a9c-487a-ad75-90bf69951aab service glance] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:19.753678 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d8839-4a9c-487a-ad75-90bf69951aab service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:19.768243 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400d8839-4a9c-487a-ad75-90bf69951aab service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:19.776333 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400d8839-4a9c-487a-ad75-90bf69951aab service glance] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:19.785263 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1189/2379] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:06:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:06:29.498767 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=59,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:06:29.781300 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=25 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:06:38.430666 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5dc0f8-da55-490e-94a4-6c171e7be2b8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:38.431241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5dc0f8-da55-490e-94a4-6c171e7be2b8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:38.431349 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5dc0f8-da55-490e-94a4-6c171e7be2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:38.473711 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0c5dc0f8-da55-490e-94a4-6c171e7be2b8 None None] MFA Rules not processed for user `6edbad98ed0f4823b9955f9f0e270149`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:06:38.475354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0c5dc0f8-da55-490e-94a4-6c171e7be2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:38.581389 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1191/2380] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 21 16:06:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:06:38.599015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400d8839-4a9c-487a-ad75-90bf69951aab service glance] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:38.601992 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-400d8839-4a9c-487a-ad75-90bf69951aab service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:38.701388 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400d8839-4a9c-487a-ad75-90bf69951aab service glance] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:38.706611 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f4d50-f5a7-42ca-b7b9-3f8beb694c8f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5bfae2cb30843b1b1a0343efef38638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:38.707377 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f4d50-f5a7-42ca-b7b9-3f8beb694c8f service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:38.707735 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f4d50-f5a7-42ca-b7b9-3f8beb694c8f service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:38.708087 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f4d50-f5a7-42ca-b7b9-3f8beb694c8f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:38.716904 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f4d50-f5a7-42ca-b7b9-3f8beb694c8f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:38.718716 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f4d50-f5a7-42ca-b7b9-3f8beb694c8f service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:38.727727 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1190/2381] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 16:06:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 16:06:40.246965 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d141d39-7a15-4bfc-84e3-2ce9962e0a4f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:40.247479 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d141d39-7a15-4bfc-84e3-2ce9962e0a4f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:40.252540 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d141d39-7a15-4bfc-84e3-2ce9962e0a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:40.283695 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2d141d39-7a15-4bfc-84e3-2ce9962e0a4f None None] MFA Rules not processed for user `6edbad98ed0f4823b9955f9f0e270149`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:06:40.286963 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2d141d39-7a15-4bfc-84e3-2ce9962e0a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:40.301333 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1192/2382] 10.222.0.52 () {58 vars in 967 bytes} [Thu Jul 21 16:06:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:06:40.319594 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f4d50-f5a7-42ca-b7b9-3f8beb694c8f service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:40.336007 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f4d50-f5a7-42ca-b7b9-3f8beb694c8f service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:40.341602 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ad467-14a2-4d68-9e9f-b01464340a36 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5bfae2cb30843b1b1a0343efef38638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:40.342426 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ad467-14a2-4d68-9e9f-b01464340a36 service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:40.342713 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ad467-14a2-4d68-9e9f-b01464340a36 service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:40.342993 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ad467-14a2-4d68-9e9f-b01464340a36 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:40.352177 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ad467-14a2-4d68-9e9f-b01464340a36 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:40.353822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ad467-14a2-4d68-9e9f-b01464340a36 service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:40.362780 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1191/2383] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 16:06:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 16:06:42.807732 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d141d39-7a15-4bfc-84e3-2ce9962e0a4f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:42.809355 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2d141d39-7a15-4bfc-84e3-2ce9962e0a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:42.911326 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d141d39-7a15-4bfc-84e3-2ce9962e0a4f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:42.916041 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0372d77-3d90-46d8-8ce0-1dea13fbcb46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:42.916930 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0372d77-3d90-46d8-8ce0-1dea13fbcb46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:42.917225 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0372d77-3d90-46d8-8ce0-1dea13fbcb46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:42.917510 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0372d77-3d90-46d8-8ce0-1dea13fbcb46 admin admin] PATH_INFO: `/v3/users/ad718e6456d542d7a2d99c549909b4c3` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:42.919776 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0372d77-3d90-46d8-8ce0-1dea13fbcb46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad718e6456d542d7a2d99c549909b4c3)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:42.922074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0372d77-3d90-46d8-8ce0-1dea13fbcb46 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:42.997180 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a0372d77-3d90-46d8-8ce0-1dea13fbcb46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad718e6456d542d7a2d99c549909b4c3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:42.998444 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a0372d77-3d90-46d8-8ce0-1dea13fbcb46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad718e6456d542d7a2d99c549909b4c3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:42.999067 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a0372d77-3d90-46d8-8ce0-1dea13fbcb46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad718e6456d542d7a2d99c549909b4c3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:43.009627 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a0372d77-3d90-46d8-8ce0-1dea13fbcb46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad718e6456d542d7a2d99c549909b4c3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:43.020511 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a0372d77-3d90-46d8-8ce0-1dea13fbcb46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad718e6456d542d7a2d99c549909b4c3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:43.061626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1193/2384] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:06:42 2022] DELETE /identity/v3/users/ad718e6456d542d7a2d99c549909b4c3 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:06:43.416343 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ad467-14a2-4d68-9e9f-b01464340a36 service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:43.458519 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ad467-14a2-4d68-9e9f-b01464340a36 service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:43.583832 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3384e-92e4-4001-9ea3-a9c4d1cd2f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:43.584977 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3384e-92e4-4001-9ea3-a9c4d1cd2f76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:43.585443 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3384e-92e4-4001-9ea3-a9c4d1cd2f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:43.585913 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3384e-92e4-4001-9ea3-a9c4d1cd2f76 admin admin] PATH_INFO: `/v3/projects/7e8fe26b25d44c89a9861a5b8e17e395` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:43.587566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3384e-92e4-4001-9ea3-a9c4d1cd2f76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e8fe26b25d44c89a9861a5b8e17e395)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:43.589907 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3384e-92e4-4001-9ea3-a9c4d1cd2f76 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:43.686154 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-08f3384e-92e4-4001-9ea3-a9c4d1cd2f76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e8fe26b25d44c89a9861a5b8e17e395', 'request_id': 'req-08f3384e-92e4-4001-9ea3-a9c4d1cd2f76'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:43.698485 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-08f3384e-92e4-4001-9ea3-a9c4d1cd2f76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e8fe26b25d44c89a9861a5b8e17e395', 'request_id': 'req-08f3384e-92e4-4001-9ea3-a9c4d1cd2f76'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:43.708273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-08f3384e-92e4-4001-9ea3-a9c4d1cd2f76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e8fe26b25d44c89a9861a5b8e17e395', 'request_id': 'req-08f3384e-92e4-4001-9ea3-a9c4d1cd2f76'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:43.725140 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-08f3384e-92e4-4001-9ea3-a9c4d1cd2f76 admin admin] The token cache is being invalidate because project 7e8fe26b25d44c89a9861a5b8e17e395 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:06:43.725738 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-08f3384e-92e4-4001-9ea3-a9c4d1cd2f76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:43.726424 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-08f3384e-92e4-4001-9ea3-a9c4d1cd2f76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:43.728242 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1192/2385] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:06:43 2022] DELETE /identity/v3/projects/7e8fe26b25d44c89a9861a5b8e17e395 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:06:43.733177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c1ef7-4cab-4701-a7d6-2d951d78228b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:43.733775 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c1ef7-4cab-4701-a7d6-2d951d78228b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:43.734319 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c1ef7-4cab-4701-a7d6-2d951d78228b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:43.757479 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c96c1ef7-4cab-4701-a7d6-2d951d78228b None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:06:43.759205 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c96c1ef7-4cab-4701-a7d6-2d951d78228b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:43.772618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:06:43.839843 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1194/2386] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:06:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:06:43.847687 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfbf638-6150-4bae-8a31-c39ae37efe51 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:43.848238 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfbf638-6150-4bae-8a31-c39ae37efe51 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:43.848524 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfbf638-6150-4bae-8a31-c39ae37efe51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:43.870334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:06:43.877994 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1cfbf638-6150-4bae-8a31-c39ae37efe51 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:06:43.881173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1cfbf638-6150-4bae-8a31-c39ae37efe51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:43.891632 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1193/2387] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:06:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:06:43.896015 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c1ef7-4cab-4701-a7d6-2d951d78228b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:43.906805 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c1ef7-4cab-4701-a7d6-2d951d78228b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:43.913207 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d60708-fd62-4a14-aa2e-299c5f297376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:43.914063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d60708-fd62-4a14-aa2e-299c5f297376 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:43.914430 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d60708-fd62-4a14-aa2e-299c5f297376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:43.914792 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d60708-fd62-4a14-aa2e-299c5f297376 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:43.915674 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d60708-fd62-4a14-aa2e-299c5f297376 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:06:43.916054 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d60708-fd62-4a14-aa2e-299c5f297376 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:43.917871 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d60708-fd62-4a14-aa2e-299c5f297376 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:43.936353 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1195/2388] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:06:43.937938 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfbf638-6150-4bae-8a31-c39ae37efe51 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:43.948070 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfbf638-6150-4bae-8a31-c39ae37efe51 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:43.953395 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142b293-cd9d-40d2-92cb-c0aee11d39c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:43.954089 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142b293-cd9d-40d2-92cb-c0aee11d39c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:43.954406 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142b293-cd9d-40d2-92cb-c0aee11d39c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:43.954686 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142b293-cd9d-40d2-92cb-c0aee11d39c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:43.955790 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d142b293-cd9d-40d2-92cb-c0aee11d39c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:43.958189 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d142b293-cd9d-40d2-92cb-c0aee11d39c7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:43.988602 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1194/2389] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:06:43 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:06:43.992958 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d60708-fd62-4a14-aa2e-299c5f297376 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:44.011127 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d60708-fd62-4a14-aa2e-299c5f297376 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:44.090529 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd614af-b2fc-4ed4-960d-363bdeda35f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:44.091493 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd614af-b2fc-4ed4-960d-363bdeda35f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:44.091947 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd614af-b2fc-4ed4-960d-363bdeda35f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:44.092402 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd614af-b2fc-4ed4-960d-363bdeda35f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:44.093112 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd614af-b2fc-4ed4-960d-363bdeda35f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:44.095134 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd614af-b2fc-4ed4-960d-363bdeda35f2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:44.127939 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1196/2390] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:06:43 2022] POST /identity/v3/users => generated 285 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:06:44.131971 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142b293-cd9d-40d2-92cb-c0aee11d39c7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:44.143487 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142b293-cd9d-40d2-92cb-c0aee11d39c7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:44.149439 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858e27a-a0ce-4a32-8b9e-33c20c211401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:44.150499 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858e27a-a0ce-4a32-8b9e-33c20c211401 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:44.150970 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858e27a-a0ce-4a32-8b9e-33c20c211401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:44.151544 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858e27a-a0ce-4a32-8b9e-33c20c211401 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:44.152443 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5858e27a-a0ce-4a32-8b9e-33c20c211401 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:44.154234 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5858e27a-a0ce-4a32-8b9e-33c20c211401 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:44.169569 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1195/2391] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:06:44.173389 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd614af-b2fc-4ed4-960d-363bdeda35f2 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:44.185297 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd614af-b2fc-4ed4-960d-363bdeda35f2 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:44.190686 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd9860-7215-4514-9ba4-16600efa7bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:44.191685 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd9860-7215-4514-9ba4-16600efa7bfa admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:44.192302 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd9860-7215-4514-9ba4-16600efa7bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:44.192785 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd9860-7215-4514-9ba4-16600efa7bfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:44.193909 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fd9860-7215-4514-9ba4-16600efa7bfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:44.196331 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fd9860-7215-4514-9ba4-16600efa7bfa admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:44.211340 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1197/2392] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:06:44.215593 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858e27a-a0ce-4a32-8b9e-33c20c211401 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:44.237370 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858e27a-a0ce-4a32-8b9e-33c20c211401 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:44.243011 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a65d4-9fb2-4816-bd54-d6b5f5b418c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:44.244660 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a65d4-9fb2-4816-bd54-d6b5f5b418c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:44.244660 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a65d4-9fb2-4816-bd54-d6b5f5b418c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:44.244660 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a65d4-9fb2-4816-bd54-d6b5f5b418c7 admin admin] PATH_INFO: `/v3/projects/91a5daa229ce476eb251a07dda8b935e/users/ac4c992462884100adf54819fccb876b/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:44.267275 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a65d4-9fb2-4816-bd54-d6b5f5b418c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91a5daa229ce476eb251a07dda8b935e, user_id=ac4c992462884100adf54819fccb876b, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:44.271273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a65d4-9fb2-4816-bd54-d6b5f5b418c7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:44.293254 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1196/2393] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:06:44 2022] PUT /identity/v3/projects/91a5daa229ce476eb251a07dda8b935e/users/ac4c992462884100adf54819fccb876b/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:06:44.303463 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e033f2-2b9d-4cc7-a38c-0debaa6a10d1 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:44.304013 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e033f2-2b9d-4cc7-a38c-0debaa6a10d1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:44.304493 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e033f2-2b9d-4cc7-a38c-0debaa6a10d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:44.353657 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-51e033f2-2b9d-4cc7-a38c-0debaa6a10d1 None None] MFA Rules not processed for user `ac4c992462884100adf54819fccb876b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:06:44.355032 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-51e033f2-2b9d-4cc7-a38c-0debaa6a10d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:44.447541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a65d4-9fb2-4816-bd54-d6b5f5b418c7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:44.449880 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9d8a65d4-9fb2-4816-bd54-d6b5f5b418c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:44.451209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1198/2394] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:06:44 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:06:44.523570 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a65d4-9fb2-4816-bd54-d6b5f5b418c7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:44.528377 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc900ae5-ec74-4ae3-a140-fd7a46be0be7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73a7d5f37e144768d7cc53f13cd5157', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:44.529107 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc900ae5-ec74-4ae3-a140-fd7a46be0be7 service glance] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:44.529465 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc900ae5-ec74-4ae3-a140-fd7a46be0be7 service glance] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:44.529846 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc900ae5-ec74-4ae3-a140-fd7a46be0be7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:44.543875 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc900ae5-ec74-4ae3-a140-fd7a46be0be7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:44.545763 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc900ae5-ec74-4ae3-a140-fd7a46be0be7 service glance] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:44.577397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1197/2395] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:06:44 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:06:54.028267 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043e94a-0c5e-43fa-ae0f-2cf58213c235 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:54.028890 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043e94a-0c5e-43fa-ae0f-2cf58213c235 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:54.029229 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043e94a-0c5e-43fa-ae0f-2cf58213c235 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:54.031394 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1199/2396] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jul 21 16:06:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 16:06:54.038403 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5ceea-26e3-4e81-b6a5-656953d51902 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:54.039297 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5ceea-26e3-4e81-b6a5-656953d51902 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:54.039580 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5ceea-26e3-4e81-b6a5-656953d51902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:54.058285 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1ea5ceea-26e3-4e81-b6a5-656953d51902 None None] MFA Rules not processed for user `092553a86f8848dba8c7271f5c0b2c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:06:54.059738 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1ea5ceea-26e3-4e81-b6a5-656953d51902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:54.133453 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1198/2397] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jul 21 16:06:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:06:54.145024 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043e94a-0c5e-43fa-ae0f-2cf58213c235 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:54.146876 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6043e94a-0c5e-43fa-ae0f-2cf58213c235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:54.160526 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043e94a-0c5e-43fa-ae0f-2cf58213c235 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:54.164336 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235b678-72dc-4c79-94be-fbdc064134ac service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73a7d5f37e144768d7cc53f13cd5157', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:54.165131 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235b678-72dc-4c79-94be-fbdc064134ac service glance] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:54.165486 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235b678-72dc-4c79-94be-fbdc064134ac service glance] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:54.165898 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235b678-72dc-4c79-94be-fbdc064134ac service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:54.177443 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4235b678-72dc-4c79-94be-fbdc064134ac service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:54.189054 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4235b678-72dc-4c79-94be-fbdc064134ac service glance] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:54.191092 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:06:54.197783 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1200/2398] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:06:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:06:54.268034 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f27c7-3cc5-4291-b44b-6cf7c2e4169d None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:54.268324 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f27c7-3cc5-4291-b44b-6cf7c2e4169d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:54.268606 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f27c7-3cc5-4291-b44b-6cf7c2e4169d None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:54.270678 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1199/2399] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jul 21 16:06:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 16:06:54.275409 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bca97-c62d-4bb6-b45a-22ed255cc29b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:54.275727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bca97-c62d-4bb6-b45a-22ed255cc29b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:54.276006 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bca97-c62d-4bb6-b45a-22ed255cc29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:54.294196 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3a8bca97-c62d-4bb6-b45a-22ed255cc29b None None] MFA Rules not processed for user `1654fdf5ba6b46ba8d9b86a83a76a4b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:06:54.295316 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3a8bca97-c62d-4bb6-b45a-22ed255cc29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:54.347663 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1201/2400] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jul 21 16:06:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:06:54.360302 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f27c7-3cc5-4291-b44b-6cf7c2e4169d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:54.362101 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-515f27c7-3cc5-4291-b44b-6cf7c2e4169d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:54.375874 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f27c7-3cc5-4291-b44b-6cf7c2e4169d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:54.379506 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577389cb-7bc1-4887-a0f2-bb4526878714 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:54.380137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577389cb-7bc1-4887-a0f2-bb4526878714 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:54.380422 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577389cb-7bc1-4887-a0f2-bb4526878714 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:54.380809 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577389cb-7bc1-4887-a0f2-bb4526878714 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:54.387863 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577389cb-7bc1-4887-a0f2-bb4526878714 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:54.389289 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577389cb-7bc1-4887-a0f2-bb4526878714 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:54.397762 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1200/2401] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:06:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:06:54.480283 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9621b79-630b-4251-a4d0-7a9f03b66b9a None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:54.480573 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9621b79-630b-4251-a4d0-7a9f03b66b9a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:54.480859 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9621b79-630b-4251-a4d0-7a9f03b66b9a None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:54.482252 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1202/2402] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jul 21 16:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 16:06:54.488181 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cdbdee-7044-48a9-808b-2e4a7b1bee8e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:54.488474 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cdbdee-7044-48a9-808b-2e4a7b1bee8e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:54.488758 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cdbdee-7044-48a9-808b-2e4a7b1bee8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:54.507205 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-81cdbdee-7044-48a9-808b-2e4a7b1bee8e None None] MFA Rules not processed for user `092553a86f8848dba8c7271f5c0b2c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:06:54.508414 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-81cdbdee-7044-48a9-808b-2e4a7b1bee8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:54.517758 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1201/2403] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jul 21 16:06:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:06:54.528578 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9621b79-630b-4251-a4d0-7a9f03b66b9a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:54.540425 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9621b79-630b-4251-a4d0-7a9f03b66b9a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:54.544300 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763284e-2573-4014-be51-9b7d197b5302 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:54.544878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763284e-2573-4014-be51-9b7d197b5302 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:54.545164 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763284e-2573-4014-be51-9b7d197b5302 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:54.545441 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763284e-2573-4014-be51-9b7d197b5302 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:54.557542 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b763284e-2573-4014-be51-9b7d197b5302 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:54.559088 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b763284e-2573-4014-be51-9b7d197b5302 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:54.570426 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1203/2404] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:06:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:06:54.962555 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cdbdee-7044-48a9-808b-2e4a7b1bee8e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:54.963955 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-81cdbdee-7044-48a9-808b-2e4a7b1bee8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:54.984806 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:06:55.031812 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cdbdee-7044-48a9-808b-2e4a7b1bee8e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:55.036980 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb46890-97cd-41b8-88d2-8ed42ec0d1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:55.038139 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb46890-97cd-41b8-88d2-8ed42ec0d1f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:55.038581 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb46890-97cd-41b8-88d2-8ed42ec0d1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:55.039161 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb46890-97cd-41b8-88d2-8ed42ec0d1f4 admin admin] PATH_INFO: `/v3/users/009030f4b9394dd08d3f661269300f1d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:55.040152 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb46890-97cd-41b8-88d2-8ed42ec0d1f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=009030f4b9394dd08d3f661269300f1d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:55.042192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb46890-97cd-41b8-88d2-8ed42ec0d1f4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:55.096615 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cfb46890-97cd-41b8-88d2-8ed42ec0d1f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '009030f4b9394dd08d3f661269300f1d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:55.099029 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cfb46890-97cd-41b8-88d2-8ed42ec0d1f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '009030f4b9394dd08d3f661269300f1d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:55.099029 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cfb46890-97cd-41b8-88d2-8ed42ec0d1f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '009030f4b9394dd08d3f661269300f1d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:55.105970 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cfb46890-97cd-41b8-88d2-8ed42ec0d1f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '009030f4b9394dd08d3f661269300f1d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:55.114243 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cfb46890-97cd-41b8-88d2-8ed42ec0d1f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '009030f4b9394dd08d3f661269300f1d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:55.132140 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1202/2405] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:06:54 2022] DELETE /identity/v3/users/009030f4b9394dd08d3f661269300f1d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:06:55.136583 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb2856-e6a4-48b0-949c-9507a92e7fb7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:55.137037 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb2856-e6a4-48b0-949c-9507a92e7fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:55.137200 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb2856-e6a4-48b0-949c-9507a92e7fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:55.154502 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-64cb2856-e6a4-48b0-949c-9507a92e7fb7 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:06:55.155698 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-64cb2856-e6a4-48b0-949c-9507a92e7fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:55.217382 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1204/2406] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:06:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:06:55.227268 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb46890-97cd-41b8-88d2-8ed42ec0d1f4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:55.228691 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cfb46890-97cd-41b8-88d2-8ed42ec0d1f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:55.296873 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb46890-97cd-41b8-88d2-8ed42ec0d1f4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:55.301552 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a159ac-7e2a-4d78-a829-ac6bfeb365e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:55.302237 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a159ac-7e2a-4d78-a829-ac6bfeb365e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:55.302522 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a159ac-7e2a-4d78-a829-ac6bfeb365e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:55.302799 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a159ac-7e2a-4d78-a829-ac6bfeb365e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:55.312432 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a159ac-7e2a-4d78-a829-ac6bfeb365e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:55.313924 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a159ac-7e2a-4d78-a829-ac6bfeb365e1 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:55.327197 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1203/2407] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:06:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:06:55.618128 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb2856-e6a4-48b0-949c-9507a92e7fb7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:55.619694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-64cb2856-e6a4-48b0-949c-9507a92e7fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:55.634548 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb2856-e6a4-48b0-949c-9507a92e7fb7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:55.639860 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1482b11-8eec-4cd4-87d6-307ba8b3d720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:55.641146 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1482b11-8eec-4cd4-87d6-307ba8b3d720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:55.641508 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1482b11-8eec-4cd4-87d6-307ba8b3d720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:55.641897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1482b11-8eec-4cd4-87d6-307ba8b3d720 admin admin] PATH_INFO: `/v3/projects/c74c7c3813da4410838333e3a8480093` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:55.643198 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1482b11-8eec-4cd4-87d6-307ba8b3d720 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c74c7c3813da4410838333e3a8480093)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:55.645795 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1482b11-8eec-4cd4-87d6-307ba8b3d720 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:55.697947 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e1482b11-8eec-4cd4-87d6-307ba8b3d720 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c74c7c3813da4410838333e3a8480093', 'request_id': 'req-e1482b11-8eec-4cd4-87d6-307ba8b3d720'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:55.704741 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e1482b11-8eec-4cd4-87d6-307ba8b3d720 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c74c7c3813da4410838333e3a8480093', 'request_id': 'req-e1482b11-8eec-4cd4-87d6-307ba8b3d720'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:55.714509 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e1482b11-8eec-4cd4-87d6-307ba8b3d720 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c74c7c3813da4410838333e3a8480093', 'request_id': 'req-e1482b11-8eec-4cd4-87d6-307ba8b3d720'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:55.729381 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e1482b11-8eec-4cd4-87d6-307ba8b3d720 admin admin] The token cache is being invalidate because project c74c7c3813da4410838333e3a8480093 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:06:55.729743 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e1482b11-8eec-4cd4-87d6-307ba8b3d720 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:55.731586 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e1482b11-8eec-4cd4-87d6-307ba8b3d720 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:06:55.733228 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1205/2408] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:06:55 2022] DELETE /identity/v3/projects/c74c7c3813da4410838333e3a8480093 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:06:55.738577 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86cbf5e-2e8e-4d73-9b0d-1cf7e6bd55c7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:55.738865 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86cbf5e-2e8e-4d73-9b0d-1cf7e6bd55c7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:55.739166 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86cbf5e-2e8e-4d73-9b0d-1cf7e6bd55c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:55.761001 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c86cbf5e-2e8e-4d73-9b0d-1cf7e6bd55c7 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:06:55.762273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c86cbf5e-2e8e-4d73-9b0d-1cf7e6bd55c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:55.799718 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:06:55.838934 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1204/2409] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:06:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:06:55.848104 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b211b-803a-4e3e-9926-bcbb4ae3cff2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:55.848405 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b211b-803a-4e3e-9926-bcbb4ae3cff2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:55.849306 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b211b-803a-4e3e-9926-bcbb4ae3cff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:55.869336 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-973b211b-803a-4e3e-9926-bcbb4ae3cff2 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:06:55.870698 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-973b211b-803a-4e3e-9926-bcbb4ae3cff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:55.879920 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1206/2410] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:06:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:06:55.884069 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86cbf5e-2e8e-4d73-9b0d-1cf7e6bd55c7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:55.900188 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86cbf5e-2e8e-4d73-9b0d-1cf7e6bd55c7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:55.909901 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d82531-c23c-4953-8d9d-7560e4b72dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:55.913767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d82531-c23c-4953-8d9d-7560e4b72dc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:55.913767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d82531-c23c-4953-8d9d-7560e4b72dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:55.913767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d82531-c23c-4953-8d9d-7560e4b72dc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:55.913767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d82531-c23c-4953-8d9d-7560e4b72dc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:06:55.913767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d82531-c23c-4953-8d9d-7560e4b72dc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:55.916219 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d82531-c23c-4953-8d9d-7560e4b72dc6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:55.933759 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1205/2411] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:06:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:06:55.938164 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b211b-803a-4e3e-9926-bcbb4ae3cff2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:55.948365 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b211b-803a-4e3e-9926-bcbb4ae3cff2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:55.953542 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25cd90-3ce5-4a25-97b0-d9626e7aa2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:55.954509 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25cd90-3ce5-4a25-97b0-d9626e7aa2a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:55.954872 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25cd90-3ce5-4a25-97b0-d9626e7aa2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:55.955352 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25cd90-3ce5-4a25-97b0-d9626e7aa2a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:55.956927 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc25cd90-3ce5-4a25-97b0-d9626e7aa2a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:55.959126 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc25cd90-3ce5-4a25-97b0-d9626e7aa2a4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:55.969841 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:06:55.995175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1207/2412] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:06:55 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:06:55.996271 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d82531-c23c-4953-8d9d-7560e4b72dc6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:56.006767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d82531-c23c-4953-8d9d-7560e4b72dc6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:56.068834 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d1658-7de4-425c-a75e-09a9b249c73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:56.069515 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d1658-7de4-425c-a75e-09a9b249c73a admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:56.070144 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d1658-7de4-425c-a75e-09a9b249c73a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:56.070144 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d1658-7de4-425c-a75e-09a9b249c73a admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:56.070878 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d1658-7de4-425c-a75e-09a9b249c73a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:56.072855 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d1658-7de4-425c-a75e-09a9b249c73a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:56.099743 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1206/2413] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:06:55 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:06:56.101039 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25cd90-3ce5-4a25-97b0-d9626e7aa2a4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:56.111545 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25cd90-3ce5-4a25-97b0-d9626e7aa2a4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:56.117777 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72502e9f-c15e-4644-95cd-262d811d1ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:56.118450 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72502e9f-c15e-4644-95cd-262d811d1ec0 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:56.118803 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72502e9f-c15e-4644-95cd-262d811d1ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:56.119016 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72502e9f-c15e-4644-95cd-262d811d1ec0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:56.119778 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72502e9f-c15e-4644-95cd-262d811d1ec0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:56.121380 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72502e9f-c15e-4644-95cd-262d811d1ec0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:56.133617 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1208/2414] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:06:56.137924 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d1658-7de4-425c-a75e-09a9b249c73a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:56.149826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d1658-7de4-425c-a75e-09a9b249c73a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:56.154990 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a08ace-ca57-4c53-8796-daa1be207c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:56.155657 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a08ace-ca57-4c53-8796-daa1be207c0d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:56.156086 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a08ace-ca57-4c53-8796-daa1be207c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:56.156368 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a08ace-ca57-4c53-8796-daa1be207c0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:56.157092 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a08ace-ca57-4c53-8796-daa1be207c0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:56.159053 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a08ace-ca57-4c53-8796-daa1be207c0d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:56.173533 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1207/2415] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:06:56.174835 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72502e9f-c15e-4644-95cd-262d811d1ec0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:56.186354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72502e9f-c15e-4644-95cd-262d811d1ec0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:56.191444 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c495a4-82ad-40a8-810a-23de097f6452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:56.192411 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c495a4-82ad-40a8-810a-23de097f6452 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:56.192763 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c495a4-82ad-40a8-810a-23de097f6452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:56.193049 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c495a4-82ad-40a8-810a-23de097f6452 admin admin] PATH_INFO: `/v3/projects/9d55db285dae4de79e1ecc442129bab5/users/762f5eba8cbe4bb9b40062e6cd1e9929/roles/54195e660fc349978b39d5794765a095` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:56.209987 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c495a4-82ad-40a8-810a-23de097f6452 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d55db285dae4de79e1ecc442129bab5, user_id=762f5eba8cbe4bb9b40062e6cd1e9929, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:56.212805 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c495a4-82ad-40a8-810a-23de097f6452 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:56.232513 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1209/2416] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:06:56 2022] PUT /identity/v3/projects/9d55db285dae4de79e1ecc442129bab5/users/762f5eba8cbe4bb9b40062e6cd1e9929/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:06:56.243472 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff926ad5-38e3-45ac-8481-bc7abaa96f08 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:56.243801 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff926ad5-38e3-45ac-8481-bc7abaa96f08 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:56.244192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff926ad5-38e3-45ac-8481-bc7abaa96f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:56.266715 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ff926ad5-38e3-45ac-8481-bc7abaa96f08 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:06:56.268230 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ff926ad5-38e3-45ac-8481-bc7abaa96f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:56.359976 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1208/2417] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:06:56.369950 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c495a4-82ad-40a8-810a-23de097f6452 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:56.373800 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f8c495a4-82ad-40a8-810a-23de097f6452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:56.476628 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c495a4-82ad-40a8-810a-23de097f6452 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:56.480515 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1deb78-e796-4cd2-b2c4-4e38a1808c8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:56.481181 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1deb78-e796-4cd2-b2c4-4e38a1808c8d service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:56.481472 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1deb78-e796-4cd2-b2c4-4e38a1808c8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:56.481788 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1deb78-e796-4cd2-b2c4-4e38a1808c8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:56.505119 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1deb78-e796-4cd2-b2c4-4e38a1808c8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:56.506793 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1deb78-e796-4cd2-b2c4-4e38a1808c8d service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:56.518727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1210/2418] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:06:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:06:57.867500 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8918676-253a-497f-bb00-5f96ac7341b7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:57.868646 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8918676-253a-497f-bb00-5f96ac7341b7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:57.869135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8918676-253a-497f-bb00-5f96ac7341b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:57.891746 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e8918676-253a-497f-bb00-5f96ac7341b7 None None] MFA Rules not processed for user `6edbad98ed0f4823b9955f9f0e270149`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:06:57.893270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e8918676-253a-497f-bb00-5f96ac7341b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:57.964223 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:06:57.986594 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1209/2419] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 21 16:06:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:06:58.006773 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1deb78-e796-4cd2-b2c4-4e38a1808c8d service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:06:58.008467 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fb1deb78-e796-4cd2-b2c4-4e38a1808c8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:06:58.124725 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1deb78-e796-4cd2-b2c4-4e38a1808c8d service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:06:58.134137 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c838e55-2b9e-4132-93e4-41721526926b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5bfae2cb30843b1b1a0343efef38638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:06:58.134137 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c838e55-2b9e-4132-93e4-41721526926b service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:06:58.134137 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c838e55-2b9e-4132-93e4-41721526926b service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:06:58.134137 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c838e55-2b9e-4132-93e4-41721526926b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:06:58.144678 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c838e55-2b9e-4132-93e4-41721526926b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:06:58.146529 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c838e55-2b9e-4132-93e4-41721526926b service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:06:58.156757 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1211/2420] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 16:06:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 16:07:02.617985 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ad8fa-11e7-45bf-89b7-d3965025571c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:02.618584 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ad8fa-11e7-45bf-89b7-d3965025571c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:02.618584 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ad8fa-11e7-45bf-89b7-d3965025571c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:02.650405 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-061ad8fa-11e7-45bf-89b7-d3965025571c None None] MFA Rules not processed for user `762f5eba8cbe4bb9b40062e6cd1e9929`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:07:02.651848 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-061ad8fa-11e7-45bf-89b7-d3965025571c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:07:02.727332 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1210/2421] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:07:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:07:02.732078 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c838e55-2b9e-4132-93e4-41721526926b service swift] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:07:02.744417 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c838e55-2b9e-4132-93e4-41721526926b service swift] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:07:02.750038 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c4114-1aa1-4312-b8eb-6b8481319051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:07:02.751216 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c4114-1aa1-4312-b8eb-6b8481319051 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:02.751733 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c4114-1aa1-4312-b8eb-6b8481319051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:02.752208 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c4114-1aa1-4312-b8eb-6b8481319051 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:02.753884 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c4114-1aa1-4312-b8eb-6b8481319051 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:07:02.756129 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c4114-1aa1-4312-b8eb-6b8481319051 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:07:02.813958 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1212/2422] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:07:02 2022] POST /identity/v3/projects => generated 377 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:07:02.825065 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ad8fa-11e7-45bf-89b7-d3965025571c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:07:02.837077 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ad8fa-11e7-45bf-89b7-d3965025571c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:07:02.910963 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9fdaeb-e84b-4bc3-8011-3e4586001360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:07:02.911961 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9fdaeb-e84b-4bc3-8011-3e4586001360 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:02.912437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9fdaeb-e84b-4bc3-8011-3e4586001360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:02.912895 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9fdaeb-e84b-4bc3-8011-3e4586001360 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:02.913849 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9fdaeb-e84b-4bc3-8011-3e4586001360 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:07:02.915991 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9fdaeb-e84b-4bc3-8011-3e4586001360 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:07:02.946372 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1211/2423] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:07:02 2022] POST /identity/v3/users => generated 295 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:07:02.950142 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c4114-1aa1-4312-b8eb-6b8481319051 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:07:02.968335 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c4114-1aa1-4312-b8eb-6b8481319051 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:07:02.974377 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a2898-dd52-4b0b-9040-34db07cf5d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:07:02.975190 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a2898-dd52-4b0b-9040-34db07cf5d20 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:02.975696 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a2898-dd52-4b0b-9040-34db07cf5d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:02.976072 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a2898-dd52-4b0b-9040-34db07cf5d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:02.976893 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a2898-dd52-4b0b-9040-34db07cf5d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:07:02.978749 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a2898-dd52-4b0b-9040-34db07cf5d20 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:07:02.993262 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1213/2424] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:07:02.997097 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9fdaeb-e84b-4bc3-8011-3e4586001360 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:07:03.008027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9fdaeb-e84b-4bc3-8011-3e4586001360 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:07:03.012798 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e26c0-5a5a-48f3-9d6f-992e611191d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:07:03.013732 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e26c0-5a5a-48f3-9d6f-992e611191d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:03.014025 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e26c0-5a5a-48f3-9d6f-992e611191d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:03.014310 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e26c0-5a5a-48f3-9d6f-992e611191d7 admin admin] PATH_INFO: `/v3/projects/5fd0067e651f464db47b7c377b40ac34/users/ceeae25b81044757a915c36b3cdd1b1e/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:03.030305 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93e26c0-5a5a-48f3-9d6f-992e611191d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fd0067e651f464db47b7c377b40ac34, user_id=ceeae25b81044757a915c36b3cdd1b1e, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:07:03.033053 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93e26c0-5a5a-48f3-9d6f-992e611191d7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:07:03.046924 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1212/2425] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:07:02 2022] PUT /identity/v3/projects/5fd0067e651f464db47b7c377b40ac34/users/ceeae25b81044757a915c36b3cdd1b1e/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:07:03.051731 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a2898-dd52-4b0b-9040-34db07cf5d20 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:07:03.061805 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a2898-dd52-4b0b-9040-34db07cf5d20 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:07:03.078023 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:07:03.128481 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe27671-4928-4bc2-a4b3-a6abee3b666d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:07:03.129163 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe27671-4928-4bc2-a4b3-a6abee3b666d admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:03.129454 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe27671-4928-4bc2-a4b3-a6abee3b666d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:03.129770 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe27671-4928-4bc2-a4b3-a6abee3b666d admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:03.130517 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe27671-4928-4bc2-a4b3-a6abee3b666d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:07:03.132181 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe27671-4928-4bc2-a4b3-a6abee3b666d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:07:03.146204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1214/2426] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:07:03.147713 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e26c0-5a5a-48f3-9d6f-992e611191d7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:07:03.158352 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e26c0-5a5a-48f3-9d6f-992e611191d7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:07:03.163729 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c308e357-293b-4ec5-ac9d-13c4a718e169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:07:03.164631 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308e357-293b-4ec5-ac9d-13c4a718e169 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:03.164915 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308e357-293b-4ec5-ac9d-13c4a718e169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:03.165192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308e357-293b-4ec5-ac9d-13c4a718e169 admin admin] PATH_INFO: `/v3/projects/5fd0067e651f464db47b7c377b40ac34/users/ceeae25b81044757a915c36b3cdd1b1e/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:03.167487 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c308e357-293b-4ec5-ac9d-13c4a718e169 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fd0067e651f464db47b7c377b40ac34, user_id=ceeae25b81044757a915c36b3cdd1b1e, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:07:03.170191 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c308e357-293b-4ec5-ac9d-13c4a718e169 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:07:03.189664 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1213/2427] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:07:03 2022] PUT /identity/v3/projects/5fd0067e651f464db47b7c377b40ac34/users/ceeae25b81044757a915c36b3cdd1b1e/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:07:09.854695 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb02d7-66ed-4142-a6dd-149c1956697a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:09.855288 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb02d7-66ed-4142-a6dd-149c1956697a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:09.855288 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb02d7-66ed-4142-a6dd-149c1956697a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:09.882053 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6feb02d7-66ed-4142-a6dd-149c1956697a None None] MFA Rules not processed for user `ceeae25b81044757a915c36b3cdd1b1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:07:09.883352 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6feb02d7-66ed-4142-a6dd-149c1956697a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:07:09.961450 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1215/2428] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:07:09 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:07:09.975145 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c308e357-293b-4ec5-ac9d-13c4a718e169 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:07:09.977265 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c308e357-293b-4ec5-ac9d-13c4a718e169 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:07:10.026306 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:07:10.076987 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c308e357-293b-4ec5-ac9d-13c4a718e169 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:07:10.083984 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bbba84-12df-4a5a-9c34-3ec533ebc8d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:07:10.084639 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbba84-12df-4a5a-9c34-3ec533ebc8d5 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:10.084922 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbba84-12df-4a5a-9c34-3ec533ebc8d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:10.085921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbba84-12df-4a5a-9c34-3ec533ebc8d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:10.148085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bbba84-12df-4a5a-9c34-3ec533ebc8d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:07:10.149895 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bbba84-12df-4a5a-9c34-3ec533ebc8d5 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:07:10.160229 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1214/2429] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:07:09 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:07:19.957665 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=56 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:07:20.155081 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=41 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:07:21.450691 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb02d7-66ed-4142-a6dd-149c1956697a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:07:21.452443 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6feb02d7-66ed-4142-a6dd-149c1956697a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:07:21.524118 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb02d7-66ed-4142-a6dd-149c1956697a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:07:21.528737 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e1b0e-29df-4d25-aba9-47e746f8b6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:07:21.529639 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e1b0e-29df-4d25-aba9-47e746f8b6a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:21.529939 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e1b0e-29df-4d25-aba9-47e746f8b6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:21.530214 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e1b0e-29df-4d25-aba9-47e746f8b6a0 admin admin] PATH_INFO: `/v3/users/ac4c992462884100adf54819fccb876b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:21.531419 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93e1b0e-29df-4d25-aba9-47e746f8b6a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac4c992462884100adf54819fccb876b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:07:21.533997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93e1b0e-29df-4d25-aba9-47e746f8b6a0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:07:21.595333 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e93e1b0e-29df-4d25-aba9-47e746f8b6a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac4c992462884100adf54819fccb876b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:07:21.596697 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e93e1b0e-29df-4d25-aba9-47e746f8b6a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac4c992462884100adf54819fccb876b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:07:21.597306 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e93e1b0e-29df-4d25-aba9-47e746f8b6a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac4c992462884100adf54819fccb876b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:07:21.605033 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e93e1b0e-29df-4d25-aba9-47e746f8b6a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac4c992462884100adf54819fccb876b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:07:21.612718 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e93e1b0e-29df-4d25-aba9-47e746f8b6a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac4c992462884100adf54819fccb876b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:07:21.630104 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1216/2430] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:07:21 2022] DELETE /identity/v3/users/ac4c992462884100adf54819fccb876b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:07:21.634516 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e7f0a-a4e9-4b80-b318-aa2aa493af84 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:21.634958 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e7f0a-a4e9-4b80-b318-aa2aa493af84 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:21.635063 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e7f0a-a4e9-4b80-b318-aa2aa493af84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:21.652675 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-161e7f0a-a4e9-4b80-b318-aa2aa493af84 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:07:21.653931 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-161e7f0a-a4e9-4b80-b318-aa2aa493af84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:07:21.712427 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1215/2431] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:07:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:07:21.721875 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e1b0e-29df-4d25-aba9-47e746f8b6a0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:07:21.723663 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e93e1b0e-29df-4d25-aba9-47e746f8b6a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:07:21.792875 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e1b0e-29df-4d25-aba9-47e746f8b6a0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:07:21.796828 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289dabd-3b81-4b9d-9ece-856765543e54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:07:21.797560 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289dabd-3b81-4b9d-9ece-856765543e54 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:21.797875 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289dabd-3b81-4b9d-9ece-856765543e54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:21.798157 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289dabd-3b81-4b9d-9ece-856765543e54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:21.807798 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289dabd-3b81-4b9d-9ece-856765543e54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:07:21.809244 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289dabd-3b81-4b9d-9ece-856765543e54 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:07:21.819275 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1217/2432] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:07:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:07:22.112347 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e7f0a-a4e9-4b80-b318-aa2aa493af84 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:07:22.113923 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-161e7f0a-a4e9-4b80-b318-aa2aa493af84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:07:22.128566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e7f0a-a4e9-4b80-b318-aa2aa493af84 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:07:22.134383 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e20ced-6841-47c1-9040-d16471e7f406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:07:22.135263 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e20ced-6841-47c1-9040-d16471e7f406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:22.135544 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e20ced-6841-47c1-9040-d16471e7f406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:22.135821 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e20ced-6841-47c1-9040-d16471e7f406 admin admin] PATH_INFO: `/v3/projects/91a5daa229ce476eb251a07dda8b935e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:22.137078 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e20ced-6841-47c1-9040-d16471e7f406 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91a5daa229ce476eb251a07dda8b935e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:07:22.139107 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e20ced-6841-47c1-9040-d16471e7f406 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:07:22.187078 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-45e20ced-6841-47c1-9040-d16471e7f406 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91a5daa229ce476eb251a07dda8b935e', 'request_id': 'req-45e20ced-6841-47c1-9040-d16471e7f406'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:07:22.193852 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-45e20ced-6841-47c1-9040-d16471e7f406 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91a5daa229ce476eb251a07dda8b935e', 'request_id': 'req-45e20ced-6841-47c1-9040-d16471e7f406'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:07:22.199375 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-45e20ced-6841-47c1-9040-d16471e7f406 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91a5daa229ce476eb251a07dda8b935e', 'request_id': 'req-45e20ced-6841-47c1-9040-d16471e7f406'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:07:22.210573 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-45e20ced-6841-47c1-9040-d16471e7f406 admin admin] The token cache is being invalidate because project 91a5daa229ce476eb251a07dda8b935e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:07:22.210864 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-45e20ced-6841-47c1-9040-d16471e7f406 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:07:22.212765 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-45e20ced-6841-47c1-9040-d16471e7f406 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:07:22.215460 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1216/2433] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:07:22 2022] DELETE /identity/v3/projects/91a5daa229ce476eb251a07dda8b935e => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:07:22.220462 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dba283-6061-4364-9d67-833c0f6f94e0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:22.220751 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dba283-6061-4364-9d67-833c0f6f94e0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:22.221032 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dba283-6061-4364-9d67-833c0f6f94e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:22.239823 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a7dba283-6061-4364-9d67-833c0f6f94e0 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:07:22.241041 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a7dba283-6061-4364-9d67-833c0f6f94e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:07:22.259581 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:07:22.304492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1218/2434] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:07:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:07:22.313185 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2d5d6-ce00-4ab7-885a-37302974ac99 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:22.313471 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2d5d6-ce00-4ab7-885a-37302974ac99 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:22.313781 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2d5d6-ce00-4ab7-885a-37302974ac99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:22.333220 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5bc2d5d6-ce00-4ab7-885a-37302974ac99 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:07:22.334442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5bc2d5d6-ce00-4ab7-885a-37302974ac99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:07:22.342254 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1217/2435] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:07:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:07:22.346382 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dba283-6061-4364-9d67-833c0f6f94e0 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:07:22.356480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dba283-6061-4364-9d67-833c0f6f94e0 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:07:22.361185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5803a2b5-7ff9-4d3f-bd05-6fdc3aacf5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:07:22.362456 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803a2b5-7ff9-4d3f-bd05-6fdc3aacf5f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:22.362766 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803a2b5-7ff9-4d3f-bd05-6fdc3aacf5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:22.363043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803a2b5-7ff9-4d3f-bd05-6fdc3aacf5f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:22.363768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5803a2b5-7ff9-4d3f-bd05-6fdc3aacf5f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:07:22.364101 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5803a2b5-7ff9-4d3f-bd05-6fdc3aacf5f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:07:22.365740 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5803a2b5-7ff9-4d3f-bd05-6fdc3aacf5f2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:07:22.377869 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1219/2436] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:07:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:07:22.381838 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2d5d6-ce00-4ab7-885a-37302974ac99 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:07:22.391425 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2d5d6-ce00-4ab7-885a-37302974ac99 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:07:22.400120 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2793db0d-bf54-4281-bc78-610b89de6bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:07:22.400120 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793db0d-bf54-4281-bc78-610b89de6bd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:22.400120 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793db0d-bf54-4281-bc78-610b89de6bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:22.400120 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793db0d-bf54-4281-bc78-610b89de6bd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:22.401172 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2793db0d-bf54-4281-bc78-610b89de6bd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:07:22.401319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2793db0d-bf54-4281-bc78-610b89de6bd0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:07:22.437757 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1218/2437] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:07:22 2022] POST /identity/v3/projects => generated 399 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:07:22.442459 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5803a2b5-7ff9-4d3f-bd05-6fdc3aacf5f2 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:07:22.452802 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5803a2b5-7ff9-4d3f-bd05-6fdc3aacf5f2 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:07:22.511200 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c63b97-078f-4d37-9767-5b7f197bd938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:07:22.512016 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c63b97-078f-4d37-9767-5b7f197bd938 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:22.512309 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c63b97-078f-4d37-9767-5b7f197bd938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:22.512618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c63b97-078f-4d37-9767-5b7f197bd938 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:22.513330 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c63b97-078f-4d37-9767-5b7f197bd938 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:07:22.515298 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c63b97-078f-4d37-9767-5b7f197bd938 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:07:22.537121 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1220/2438] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:07:22 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:07:22.541376 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2793db0d-bf54-4281-bc78-610b89de6bd0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:07:22.552719 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2793db0d-bf54-4281-bc78-610b89de6bd0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:07:22.557849 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1300473-6fda-4a26-8fa8-88fb4f71869f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:07:22.558550 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1300473-6fda-4a26-8fa8-88fb4f71869f admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:22.559824 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1300473-6fda-4a26-8fa8-88fb4f71869f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:22.560104 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1300473-6fda-4a26-8fa8-88fb4f71869f admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:22.560867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1300473-6fda-4a26-8fa8-88fb4f71869f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:07:22.562553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1300473-6fda-4a26-8fa8-88fb4f71869f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:07:22.574864 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1219/2439] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 16:07:22.579012 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c63b97-078f-4d37-9767-5b7f197bd938 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:07:22.589769 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c63b97-078f-4d37-9767-5b7f197bd938 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:07:22.594418 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4182cc21-830c-4906-85a9-c163c10e438d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:07:22.595200 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182cc21-830c-4906-85a9-c163c10e438d admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:22.595487 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182cc21-830c-4906-85a9-c163c10e438d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:22.596251 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182cc21-830c-4906-85a9-c163c10e438d admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:22.596983 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4182cc21-830c-4906-85a9-c163c10e438d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:07:22.598868 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4182cc21-830c-4906-85a9-c163c10e438d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:07:22.609051 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1221/2440] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 16:07:22.613831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1300473-6fda-4a26-8fa8-88fb4f71869f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:07:22.623567 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1300473-6fda-4a26-8fa8-88fb4f71869f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:07:22.628257 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed46c8-dfcd-4957-9785-eb43899b2afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:07:22.629234 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed46c8-dfcd-4957-9785-eb43899b2afe admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:22.629517 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed46c8-dfcd-4957-9785-eb43899b2afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:22.629882 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed46c8-dfcd-4957-9785-eb43899b2afe admin admin] PATH_INFO: `/v3/projects/a0b0368521684bdeaa07f2c0a146d3df/users/db502d05521549a48855a8a83ed7b029/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:22.646014 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ed46c8-dfcd-4957-9785-eb43899b2afe admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0b0368521684bdeaa07f2c0a146d3df, user_id=db502d05521549a48855a8a83ed7b029, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:07:22.648736 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ed46c8-dfcd-4957-9785-eb43899b2afe admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:07:22.663002 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1220/2441] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:07:22 2022] PUT /identity/v3/projects/a0b0368521684bdeaa07f2c0a146d3df/users/db502d05521549a48855a8a83ed7b029/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:07:22.673073 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72862a86-b371-41ce-9755-ff00cd9c1339 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:22.673368 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72862a86-b371-41ce-9755-ff00cd9c1339 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:22.673681 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72862a86-b371-41ce-9755-ff00cd9c1339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:22.706189 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-72862a86-b371-41ce-9755-ff00cd9c1339 None None] MFA Rules not processed for user `db502d05521549a48855a8a83ed7b029`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:07:22.707510 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-72862a86-b371-41ce-9755-ff00cd9c1339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:07:22.772246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1222/2442] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:07:22 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:07:22.784010 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed46c8-dfcd-4957-9785-eb43899b2afe admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:07:22.785718 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-25ed46c8-dfcd-4957-9785-eb43899b2afe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:07:22.807508 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:07:22.844385 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed46c8-dfcd-4957-9785-eb43899b2afe admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:07:22.848298 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf4a88-3ac0-4b2e-b76b-c5431fd6bb1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:07:22.848935 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf4a88-3ac0-4b2e-b76b-c5431fd6bb1a service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:22.849214 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf4a88-3ac0-4b2e-b76b-c5431fd6bb1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:22.849496 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf4a88-3ac0-4b2e-b76b-c5431fd6bb1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:22.857842 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf4a88-3ac0-4b2e-b76b-c5431fd6bb1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:07:22.859276 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf4a88-3ac0-4b2e-b76b-c5431fd6bb1a service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:07:22.868517 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1221/2443] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:07:22 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:07:31.111937 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72862a86-b371-41ce-9755-ff00cd9c1339 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:07:31.130712 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:07:31.177079 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72862a86-b371-41ce-9755-ff00cd9c1339 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:07:31.181565 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55843dc-ece2-405c-adb2-cef068c615ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:07:31.182451 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55843dc-ece2-405c-adb2-cef068c615ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:31.182738 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55843dc-ece2-405c-adb2-cef068c615ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:31.183019 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55843dc-ece2-405c-adb2-cef068c615ef admin admin] PATH_INFO: `/v3/users/db502d05521549a48855a8a83ed7b029` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:31.184200 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55843dc-ece2-405c-adb2-cef068c615ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=db502d05521549a48855a8a83ed7b029)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:07:31.186207 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55843dc-ece2-405c-adb2-cef068c615ef admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:07:31.233484 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f55843dc-ece2-405c-adb2-cef068c615ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db502d05521549a48855a8a83ed7b029'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:07:31.234172 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f55843dc-ece2-405c-adb2-cef068c615ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db502d05521549a48855a8a83ed7b029'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:07:31.234783 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f55843dc-ece2-405c-adb2-cef068c615ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db502d05521549a48855a8a83ed7b029'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:07:31.242660 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f55843dc-ece2-405c-adb2-cef068c615ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db502d05521549a48855a8a83ed7b029'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:07:31.250104 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f55843dc-ece2-405c-adb2-cef068c615ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db502d05521549a48855a8a83ed7b029'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:07:31.270945 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1223/2444] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:07:31 2022] DELETE /identity/v3/users/db502d05521549a48855a8a83ed7b029 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:07:31.275226 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84e2f9-4287-4c08-a5e1-3eb306208798 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:31.275748 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84e2f9-4287-4c08-a5e1-3eb306208798 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:31.275862 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84e2f9-4287-4c08-a5e1-3eb306208798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:31.293498 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-be84e2f9-4287-4c08-a5e1-3eb306208798 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:07:31.295037 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-be84e2f9-4287-4c08-a5e1-3eb306208798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:07:31.359454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1222/2445] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:07:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:07:31.369881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55843dc-ece2-405c-adb2-cef068c615ef admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:07:31.370980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f55843dc-ece2-405c-adb2-cef068c615ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:07:31.433483 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55843dc-ece2-405c-adb2-cef068c615ef admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:07:31.437264 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac4683b-8674-45fb-8b33-82df2c21cef0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:07:31.437985 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4683b-8674-45fb-8b33-82df2c21cef0 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:31.438275 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4683b-8674-45fb-8b33-82df2c21cef0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:31.438558 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4683b-8674-45fb-8b33-82df2c21cef0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:31.449263 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac4683b-8674-45fb-8b33-82df2c21cef0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:07:31.450819 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac4683b-8674-45fb-8b33-82df2c21cef0 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:07:31.462542 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1224/2446] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:07:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:07:31.744203 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be84e2f9-4287-4c08-a5e1-3eb306208798 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:07:31.745668 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-be84e2f9-4287-4c08-a5e1-3eb306208798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:07:31.758006 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be84e2f9-4287-4c08-a5e1-3eb306208798 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:07:31.762752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a4e62-b958-441e-bb18-c495e525ef6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:07:31.763613 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a4e62-b958-441e-bb18-c495e525ef6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:31.763896 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a4e62-b958-441e-bb18-c495e525ef6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:31.764171 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a4e62-b958-441e-bb18-c495e525ef6b admin admin] PATH_INFO: `/v3/projects/a0b0368521684bdeaa07f2c0a146d3df` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:31.765383 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1a4e62-b958-441e-bb18-c495e525ef6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0b0368521684bdeaa07f2c0a146d3df)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:07:31.767390 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1a4e62-b958-441e-bb18-c495e525ef6b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:07:31.819203 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1c1a4e62-b958-441e-bb18-c495e525ef6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0b0368521684bdeaa07f2c0a146d3df', 'request_id': 'req-1c1a4e62-b958-441e-bb18-c495e525ef6b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:07:31.827963 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1c1a4e62-b958-441e-bb18-c495e525ef6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0b0368521684bdeaa07f2c0a146d3df', 'request_id': 'req-1c1a4e62-b958-441e-bb18-c495e525ef6b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:07:31.836781 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1c1a4e62-b958-441e-bb18-c495e525ef6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0b0368521684bdeaa07f2c0a146d3df', 'request_id': 'req-1c1a4e62-b958-441e-bb18-c495e525ef6b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:07:31.867560 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1c1a4e62-b958-441e-bb18-c495e525ef6b admin admin] The token cache is being invalidate because project a0b0368521684bdeaa07f2c0a146d3df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:07:31.867755 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1c1a4e62-b958-441e-bb18-c495e525ef6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:07:31.868402 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1c1a4e62-b958-441e-bb18-c495e525ef6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:07:31.870252 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1223/2447] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:07:31 2022] DELETE /identity/v3/projects/a0b0368521684bdeaa07f2c0a146d3df => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:07:32.507089 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e31a2-9e78-4332-878a-98a5f645d2e9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:32.507650 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e31a2-9e78-4332-878a-98a5f645d2e9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:32.507936 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e31a2-9e78-4332-878a-98a5f645d2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:32.525937 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b19e31a2-9e78-4332-878a-98a5f645d2e9 None None] MFA Rules not processed for user `6edbad98ed0f4823b9955f9f0e270149`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:07:32.527160 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b19e31a2-9e78-4332-878a-98a5f645d2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:07:32.565565 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:07:32.589197 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1225/2448] 10.222.0.52 () {58 vars in 967 bytes} [Thu Jul 21 16:07:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:07:32.612116 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a4e62-b958-441e-bb18-c495e525ef6b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:07:32.613524 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1c1a4e62-b958-441e-bb18-c495e525ef6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:07:32.662726 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:07:32.689969 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a4e62-b958-441e-bb18-c495e525ef6b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:07:32.695319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9173539-d7fe-4adf-b3b4-5a6929c514c6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5bfae2cb30843b1b1a0343efef38638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:07:32.696127 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9173539-d7fe-4adf-b3b4-5a6929c514c6 service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:32.696413 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9173539-d7fe-4adf-b3b4-5a6929c514c6 service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:32.696690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9173539-d7fe-4adf-b3b4-5a6929c514c6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:32.704897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9173539-d7fe-4adf-b3b4-5a6929c514c6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:07:32.706522 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9173539-d7fe-4adf-b3b4-5a6929c514c6 service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:07:32.715011 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1224/2449] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 16:07:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 16:07:33.008711 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2071b-293d-4e54-83b6-4e10d367657f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:33.009005 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2071b-293d-4e54-83b6-4e10d367657f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:33.009290 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2071b-293d-4e54-83b6-4e10d367657f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:33.028638 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d1a2071b-293d-4e54-83b6-4e10d367657f None None] MFA Rules not processed for user `762f5eba8cbe4bb9b40062e6cd1e9929`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:07:33.029932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d1a2071b-293d-4e54-83b6-4e10d367657f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:07:33.095808 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1226/2450] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:07:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:07:35.848258 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9173539-d7fe-4adf-b3b4-5a6929c514c6 service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:07:35.849667 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f9173539-d7fe-4adf-b3b4-5a6929c514c6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:07:35.920588 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9173539-d7fe-4adf-b3b4-5a6929c514c6 service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:07:35.926271 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686da9d7-fded-4863-b648-ddf40ef766ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822d9d3521db4a9688f51c6313c7dd74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:07:35.927000 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686da9d7-fded-4863-b648-ddf40ef766ce service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:35.927258 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686da9d7-fded-4863-b648-ddf40ef766ce service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:35.927583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686da9d7-fded-4863-b648-ddf40ef766ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:35.929553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-686da9d7-fded-4863-b648-ddf40ef766ce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:07:35.942315 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686da9d7-fded-4863-b648-ddf40ef766ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:07:35.943918 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686da9d7-fded-4863-b648-ddf40ef766ce service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:07:35.954526 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1225/2451] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:07:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:07:40.156468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a302ad-d9e5-4771-9653-e79be438e03e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:40.156985 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a302ad-d9e5-4771-9653-e79be438e03e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:40.157084 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a302ad-d9e5-4771-9653-e79be438e03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:40.176070 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-14a302ad-d9e5-4771-9653-e79be438e03e None None] MFA Rules not processed for user `57361619d5c34441b16ad617a4fe22d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:07:40.177355 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-14a302ad-d9e5-4771-9653-e79be438e03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:07:40.231936 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1227/2452] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:07:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:07:45.949846 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=43 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:07:50.225365 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=57 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:07:52.119896 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686da9d7-fded-4863-b648-ddf40ef766ce service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:07:52.121343 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-686da9d7-fded-4863-b648-ddf40ef766ce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:07:52.175750 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686da9d7-fded-4863-b648-ddf40ef766ce service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:07:52.179246 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d533bce-95eb-4df7-ae1a-b6e51557023b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:07:52.179876 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d533bce-95eb-4df7-ae1a-b6e51557023b service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:52.180157 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d533bce-95eb-4df7-ae1a-b6e51557023b service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:52.180432 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d533bce-95eb-4df7-ae1a-b6e51557023b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:52.182209 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4d533bce-95eb-4df7-ae1a-b6e51557023b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:07:52.244055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d533bce-95eb-4df7-ae1a-b6e51557023b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:07:52.245514 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d533bce-95eb-4df7-ae1a-b6e51557023b service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:07:52.256004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1226/2453] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:07:52 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:07:57.778746 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef520f-2766-4845-ba47-38dd3461e964 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:07:57.779236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef520f-2766-4845-ba47-38dd3461e964 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:07:57.779353 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef520f-2766-4845-ba47-38dd3461e964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:07:57.796333 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3eef520f-2766-4845-ba47-38dd3461e964 None None] MFA Rules not processed for user `a50cbf92286c420f9bee6e05c33dc440`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:07:57.797595 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3eef520f-2766-4845-ba47-38dd3461e964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:07:57.851217 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1228/2454] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:07:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:08:02.250856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=50 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:08:07.843855 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=25 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:08:29.384531 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742fdd3-7e52-415c-896c-46212ebfb8b8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:08:29.385018 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742fdd3-7e52-415c-896c-46212ebfb8b8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:08:29.385143 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742fdd3-7e52-415c-896c-46212ebfb8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:08:29.403283 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0742fdd3-7e52-415c-896c-46212ebfb8b8 None None] MFA Rules not processed for user `3f984e42c1d3464f94ea9693a9eeb55a`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:08:29.404534 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0742fdd3-7e52-415c-896c-46212ebfb8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:08:29.461975 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1227/2455] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:08:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:08:39.454985 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=25 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:09:10.759993 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805dc157-d6a5-4cdb-aa30-65be5bac7bf7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:09:10.760511 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805dc157-d6a5-4cdb-aa30-65be5bac7bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:09:10.760604 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805dc157-d6a5-4cdb-aa30-65be5bac7bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:09:10.778211 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-805dc157-d6a5-4cdb-aa30-65be5bac7bf7 None None] MFA Rules not processed for user `57361619d5c34441b16ad617a4fe22d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:09:10.779430 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-805dc157-d6a5-4cdb-aa30-65be5bac7bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:09:10.786447 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1229/2456] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:09:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:09:17.002680 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414a94a-55d3-406e-a983-877c431f920f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:09:17.003811 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414a94a-55d3-406e-a983-877c431f920f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:09:17.004098 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414a94a-55d3-406e-a983-877c431f920f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:09:17.022418 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0414a94a-55d3-406e-a983-877c431f920f None None] MFA Rules not processed for user `a50cbf92286c420f9bee6e05c33dc440`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:09:17.023770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0414a94a-55d3-406e-a983-877c431f920f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:09:17.030932 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1228/2457] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:09:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:09:20.773076 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:09:27.016932 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:09:27.643519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805dc157-d6a5-4cdb-aa30-65be5bac7bf7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:09:27.653040 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805dc157-d6a5-4cdb-aa30-65be5bac7bf7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:09:27.657165 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ed037-4ac3-4188-b7a0-aa6d594a3662 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:09:27.657834 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ed037-4ac3-4188-b7a0-aa6d594a3662 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:09:27.658122 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ed037-4ac3-4188-b7a0-aa6d594a3662 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:09:27.658401 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ed037-4ac3-4188-b7a0-aa6d594a3662 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:09:27.660173 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8e8ed037-4ac3-4188-b7a0-aa6d594a3662 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:09:27.730384 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8ed037-4ac3-4188-b7a0-aa6d594a3662 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:09:27.731822 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8ed037-4ac3-4188-b7a0-aa6d594a3662 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:09:27.742703 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1230/2458] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:09:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:09:29.151614 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0414a94a-55d3-406e-a983-877c431f920f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:09:29.164775 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0414a94a-55d3-406e-a983-877c431f920f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:09:29.168353 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd0cc9-8ffa-4bf0-a525-59d84591f9fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:09:29.169079 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd0cc9-8ffa-4bf0-a525-59d84591f9fd service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:09:29.169395 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd0cc9-8ffa-4bf0-a525-59d84591f9fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:09:29.169704 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd0cc9-8ffa-4bf0-a525-59d84591f9fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:09:29.171485 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-11bd0cc9-8ffa-4bf0-a525-59d84591f9fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:09:29.189442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd0cc9-8ffa-4bf0-a525-59d84591f9fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:09:29.190938 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd0cc9-8ffa-4bf0-a525-59d84591f9fd service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:09:29.203248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1229/2459] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:09:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:09:37.737902 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=34 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:09:39.197437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:09:39.474196 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ed037-4ac3-4188-b7a0-aa6d594a3662 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:09:39.475678 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8e8ed037-4ac3-4188-b7a0-aa6d594a3662 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:09:39.489216 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ed037-4ac3-4188-b7a0-aa6d594a3662 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:09:39.494670 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a08cd-a500-4a4f-993d-98f80883867f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:09:39.495306 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a08cd-a500-4a4f-993d-98f80883867f service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:09:39.495590 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a08cd-a500-4a4f-993d-98f80883867f service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:09:39.495867 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a08cd-a500-4a4f-993d-98f80883867f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:09:39.497531 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5b7a08cd-a500-4a4f-993d-98f80883867f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:09:39.507524 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a08cd-a500-4a4f-993d-98f80883867f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:09:39.509185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a08cd-a500-4a4f-993d-98f80883867f service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:09:39.518586 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1231/2460] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:09:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:09:42.628870 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd580b-bf61-4649-9fc6-bfce567c8239 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:09:42.629365 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd580b-bf61-4649-9fc6-bfce567c8239 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:09:42.629467 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd580b-bf61-4649-9fc6-bfce567c8239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:09:42.648673 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-cfbd580b-bf61-4649-9fc6-bfce567c8239 None None] MFA Rules not processed for user `6edbad98ed0f4823b9955f9f0e270149`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:09:42.649874 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cfbd580b-bf61-4649-9fc6-bfce567c8239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:09:42.657793 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1230/2461] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 21 16:09:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:09:42.710010 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a08cd-a500-4a4f-993d-98f80883867f service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:09:42.721275 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a08cd-a500-4a4f-993d-98f80883867f service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:09:42.726554 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4870ea68-84f9-4768-927d-88b6318c4485 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5bfae2cb30843b1b1a0343efef38638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:09:42.727207 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870ea68-84f9-4768-927d-88b6318c4485 service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:09:42.727490 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870ea68-84f9-4768-927d-88b6318c4485 service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:09:42.727769 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870ea68-84f9-4768-927d-88b6318c4485 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:09:42.736559 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4870ea68-84f9-4768-927d-88b6318c4485 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:09:42.738221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4870ea68-84f9-4768-927d-88b6318c4485 service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:09:42.747824 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1232/2462] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 16:09:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 16:09:51.994205 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd580b-bf61-4649-9fc6-bfce567c8239 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:09:51.995731 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cfbd580b-bf61-4649-9fc6-bfce567c8239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:09:52.007612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd580b-bf61-4649-9fc6-bfce567c8239 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:09:52.011158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362366fe-6c36-4a68-8a67-45d1a7e8c08f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:09:52.012443 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362366fe-6c36-4a68-8a67-45d1a7e8c08f service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:09:52.012725 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362366fe-6c36-4a68-8a67-45d1a7e8c08f service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:09:52.013000 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362366fe-6c36-4a68-8a67-45d1a7e8c08f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:09:52.014737 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-362366fe-6c36-4a68-8a67-45d1a7e8c08f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:09:52.024253 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362366fe-6c36-4a68-8a67-45d1a7e8c08f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:09:52.025731 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362366fe-6c36-4a68-8a67-45d1a7e8c08f service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:09:52.034253 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1231/2463] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:09:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:09:52.743277 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=12 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:09:55.825039 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4870ea68-84f9-4768-927d-88b6318c4485 service swift] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:09:55.834378 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4870ea68-84f9-4768-927d-88b6318c4485 service swift] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:09:55.837915 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07834418-1997-4290-abb4-4207c030cc66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:09:55.838543 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07834418-1997-4290-abb4-4207c030cc66 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:09:55.838827 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07834418-1997-4290-abb4-4207c030cc66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:09:55.839106 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07834418-1997-4290-abb4-4207c030cc66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:09:55.840845 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-07834418-1997-4290-abb4-4207c030cc66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:09:55.850422 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07834418-1997-4290-abb4-4207c030cc66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:09:55.851842 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07834418-1997-4290-abb4-4207c030cc66 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:09:55.860229 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1233/2464] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:09:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:09:56.160211 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362366fe-6c36-4a68-8a67-45d1a7e8c08f service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:09:56.173488 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362366fe-6c36-4a68-8a67-45d1a7e8c08f service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:09:56.179222 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ece5b-d395-4f4f-93d0-fe9993eaac6b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822d9d3521db4a9688f51c6313c7dd74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:09:56.179902 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ece5b-d395-4f4f-93d0-fe9993eaac6b service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:09:56.180192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ece5b-d395-4f4f-93d0-fe9993eaac6b service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:09:56.180468 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ece5b-d395-4f4f-93d0-fe9993eaac6b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:09:56.182243 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4c2ece5b-d395-4f4f-93d0-fe9993eaac6b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:09:56.194959 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ece5b-d395-4f4f-93d0-fe9993eaac6b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:09:56.196622 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ece5b-d395-4f4f-93d0-fe9993eaac6b service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:09:56.207782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1232/2465] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:09:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:10:00.812665 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07834418-1997-4290-abb4-4207c030cc66 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:00.814124 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-07834418-1997-4290-abb4-4207c030cc66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:10:00.831457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07834418-1997-4290-abb4-4207c030cc66 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:00.838228 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9970ee58-32c4-42a6-882e-7f5a37eef9e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:00.838753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970ee58-32c4-42a6-882e-7f5a37eef9e7 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:00.839108 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970ee58-32c4-42a6-882e-7f5a37eef9e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:00.839979 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970ee58-32c4-42a6-882e-7f5a37eef9e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:00.841987 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9970ee58-32c4-42a6-882e-7f5a37eef9e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:10:00.856539 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9970ee58-32c4-42a6-882e-7f5a37eef9e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:00.858598 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9970ee58-32c4-42a6-882e-7f5a37eef9e7 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:00.873786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1234/2466] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:10:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:10:02.458343 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8710f6-e8e5-4f82-98a1-97715339adf9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:02.458878 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8710f6-e8e5-4f82-98a1-97715339adf9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:02.458979 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8710f6-e8e5-4f82-98a1-97715339adf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:02.490079 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6e8710f6-e8e5-4f82-98a1-97715339adf9 None None] MFA Rules not processed for user `6edbad98ed0f4823b9955f9f0e270149`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:10:02.491638 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6e8710f6-e8e5-4f82-98a1-97715339adf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:10:02.499315 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1233/2467] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 21 16:10:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:10:02.514867 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9970ee58-32c4-42a6-882e-7f5a37eef9e7 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:02.533012 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9970ee58-32c4-42a6-882e-7f5a37eef9e7 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:02.539583 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2636f7-93b9-4f35-b91f-966037617521 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5bfae2cb30843b1b1a0343efef38638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:02.540262 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2636f7-93b9-4f35-b91f-966037617521 service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:02.540544 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2636f7-93b9-4f35-b91f-966037617521 service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:02.540823 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2636f7-93b9-4f35-b91f-966037617521 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:02.550221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2636f7-93b9-4f35-b91f-966037617521 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:02.551816 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2636f7-93b9-4f35-b91f-966037617521 service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:02.560177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1235/2468] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 16:10:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 16:10:12.484470 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=18 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:10:12.556770 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=18 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:10:14.107024 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8710f6-e8e5-4f82-98a1-97715339adf9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:14.119646 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8710f6-e8e5-4f82-98a1-97715339adf9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:14.125039 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a7ab5b-93a7-44a3-9b4d-6a4f0c8c2c82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:14.125714 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a7ab5b-93a7-44a3-9b4d-6a4f0c8c2c82 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:14.126000 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a7ab5b-93a7-44a3-9b4d-6a4f0c8c2c82 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:14.126278 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a7ab5b-93a7-44a3-9b4d-6a4f0c8c2c82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:14.128021 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-93a7ab5b-93a7-44a3-9b4d-6a4f0c8c2c82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:10:14.138833 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a7ab5b-93a7-44a3-9b4d-6a4f0c8c2c82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:14.140419 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a7ab5b-93a7-44a3-9b4d-6a4f0c8c2c82 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:14.149401 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1234/2469] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:10:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:10:15.656813 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677b7a6-8792-4a52-9863-7d939ecd1e86 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:15.657310 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677b7a6-8792-4a52-9863-7d939ecd1e86 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:15.657421 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677b7a6-8792-4a52-9863-7d939ecd1e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:15.675950 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5677b7a6-8792-4a52-9863-7d939ecd1e86 None None] MFA Rules not processed for user `762f5eba8cbe4bb9b40062e6cd1e9929`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:10:15.677173 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5677b7a6-8792-4a52-9863-7d939ecd1e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:10:15.684762 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1236/2470] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:10:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:10:16.062103 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a7ab5b-93a7-44a3-9b4d-6a4f0c8c2c82 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:16.072091 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a7ab5b-93a7-44a3-9b4d-6a4f0c8c2c82 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:16.076681 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fec985b-80e5-4804-88fb-4d27ab396ef0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:16.077415 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec985b-80e5-4804-88fb-4d27ab396ef0 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:16.077805 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec985b-80e5-4804-88fb-4d27ab396ef0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:16.078153 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec985b-80e5-4804-88fb-4d27ab396ef0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:16.080137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4fec985b-80e5-4804-88fb-4d27ab396ef0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:10:16.090933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fec985b-80e5-4804-88fb-4d27ab396ef0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:16.092425 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fec985b-80e5-4804-88fb-4d27ab396ef0 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:16.101125 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1235/2471] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:10:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:10:19.058029 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5677b7a6-8792-4a52-9863-7d939ecd1e86 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:19.073976 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5677b7a6-8792-4a52-9863-7d939ecd1e86 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:19.080675 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae160c3-0679-481d-b551-1f743735d545 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:19.081340 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae160c3-0679-481d-b551-1f743735d545 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:19.081666 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae160c3-0679-481d-b551-1f743735d545 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:19.081945 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae160c3-0679-481d-b551-1f743735d545 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:19.083870 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5ae160c3-0679-481d-b551-1f743735d545 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:10:19.097897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae160c3-0679-481d-b551-1f743735d545 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:19.099492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae160c3-0679-481d-b551-1f743735d545 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:19.112120 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1237/2472] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:10:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:10:21.608284 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d61f1-3dee-4c7a-a67c-f179032077d6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:21.609189 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d61f1-3dee-4c7a-a67c-f179032077d6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:21.609488 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d61f1-3dee-4c7a-a67c-f179032077d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:21.628064 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-095d61f1-3dee-4c7a-a67c-f179032077d6 None None] MFA Rules not processed for user `6edbad98ed0f4823b9955f9f0e270149`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:10:21.629281 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-095d61f1-3dee-4c7a-a67c-f179032077d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:10:21.636585 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1236/2473] 10.222.0.52 () {58 vars in 967 bytes} [Thu Jul 21 16:10:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:10:21.649285 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae160c3-0679-481d-b551-1f743735d545 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:21.660459 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae160c3-0679-481d-b551-1f743735d545 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:21.665354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717877c4-f6db-41c3-93d6-8734c33b6570 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5bfae2cb30843b1b1a0343efef38638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:21.666029 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717877c4-f6db-41c3-93d6-8734c33b6570 service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:21.666316 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717877c4-f6db-41c3-93d6-8734c33b6570 service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:21.667134 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717877c4-f6db-41c3-93d6-8734c33b6570 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:21.675761 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717877c4-f6db-41c3-93d6-8734c33b6570 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:21.677317 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717877c4-f6db-41c3-93d6-8734c33b6570 service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:21.696724 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1238/2474] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 16:10:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 16:10:22.665205 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d61f1-3dee-4c7a-a67c-f179032077d6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:22.684652 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d61f1-3dee-4c7a-a67c-f179032077d6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:22.693074 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df067852-1de6-4335-a29b-e764fe3fb31a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:22.694017 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df067852-1de6-4335-a29b-e764fe3fb31a service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:22.695027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df067852-1de6-4335-a29b-e764fe3fb31a service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:22.695313 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df067852-1de6-4335-a29b-e764fe3fb31a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:22.697610 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-df067852-1de6-4335-a29b-e764fe3fb31a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:10:22.712628 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df067852-1de6-4335-a29b-e764fe3fb31a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:22.714456 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df067852-1de6-4335-a29b-e764fe3fb31a service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:22.729089 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1237/2475] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:10:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:10:25.912513 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717877c4-f6db-41c3-93d6-8734c33b6570 service swift] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:25.913977 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-717877c4-f6db-41c3-93d6-8734c33b6570 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:10:25.980395 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717877c4-f6db-41c3-93d6-8734c33b6570 service swift] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:25.984865 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10843240-849a-4d74-a51b-423d1f96f710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:25.985767 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10843240-849a-4d74-a51b-423d1f96f710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:25.986064 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10843240-849a-4d74-a51b-423d1f96f710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:25.986347 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10843240-849a-4d74-a51b-423d1f96f710 admin admin] PATH_INFO: `/v3/users/57361619d5c34441b16ad617a4fe22d8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:25.987530 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10843240-849a-4d74-a51b-423d1f96f710 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57361619d5c34441b16ad617a4fe22d8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:25.989455 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10843240-849a-4d74-a51b-423d1f96f710 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:26.035608 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-10843240-849a-4d74-a51b-423d1f96f710 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57361619d5c34441b16ad617a4fe22d8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:10:26.036253 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-10843240-849a-4d74-a51b-423d1f96f710 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57361619d5c34441b16ad617a4fe22d8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:10:26.036853 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-10843240-849a-4d74-a51b-423d1f96f710 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57361619d5c34441b16ad617a4fe22d8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:10:26.044308 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-10843240-849a-4d74-a51b-423d1f96f710 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57361619d5c34441b16ad617a4fe22d8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:10:26.051545 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-10843240-849a-4d74-a51b-423d1f96f710 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57361619d5c34441b16ad617a4fe22d8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:10:26.070905 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1239/2476] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:10:25 2022] DELETE /identity/v3/users/57361619d5c34441b16ad617a4fe22d8 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:10:26.253247 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df067852-1de6-4335-a29b-e764fe3fb31a service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:26.263966 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df067852-1de6-4335-a29b-e764fe3fb31a service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:26.330995 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8100fe0-96e8-48bf-bbf7-dac7edb0c935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:26.331862 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8100fe0-96e8-48bf-bbf7-dac7edb0c935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:26.332144 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8100fe0-96e8-48bf-bbf7-dac7edb0c935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:26.332556 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8100fe0-96e8-48bf-bbf7-dac7edb0c935 admin admin] PATH_INFO: `/v3/projects/b31d4be9786d474fa957cee32e1475ad` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:26.333819 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8100fe0-96e8-48bf-bbf7-dac7edb0c935 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b31d4be9786d474fa957cee32e1475ad)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:26.335757 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8100fe0-96e8-48bf-bbf7-dac7edb0c935 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:26.383204 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b8100fe0-96e8-48bf-bbf7-dac7edb0c935 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b31d4be9786d474fa957cee32e1475ad', 'request_id': 'req-b8100fe0-96e8-48bf-bbf7-dac7edb0c935'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:10:26.391081 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b8100fe0-96e8-48bf-bbf7-dac7edb0c935 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b31d4be9786d474fa957cee32e1475ad', 'request_id': 'req-b8100fe0-96e8-48bf-bbf7-dac7edb0c935'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:10:26.396846 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b8100fe0-96e8-48bf-bbf7-dac7edb0c935 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b31d4be9786d474fa957cee32e1475ad', 'request_id': 'req-b8100fe0-96e8-48bf-bbf7-dac7edb0c935'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:10:26.408345 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b8100fe0-96e8-48bf-bbf7-dac7edb0c935 admin admin] The token cache is being invalidate because project b31d4be9786d474fa957cee32e1475ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:10:26.408633 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b8100fe0-96e8-48bf-bbf7-dac7edb0c935 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:10:26.409267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b8100fe0-96e8-48bf-bbf7-dac7edb0c935 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:10:26.410808 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1238/2477] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:10:26 2022] DELETE /identity/v3/projects/b31d4be9786d474fa957cee32e1475ad => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:10:26.415977 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65310b-eaa0-4240-b63f-8e5be3e98b50 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:26.416274 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65310b-eaa0-4240-b63f-8e5be3e98b50 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:26.417171 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65310b-eaa0-4240-b63f-8e5be3e98b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:26.435069 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1b65310b-eaa0-4240-b63f-8e5be3e98b50 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:10:26.436275 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1b65310b-eaa0-4240-b63f-8e5be3e98b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:10:26.477258 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:10:26.500705 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1240/2478] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:10:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:10:26.508993 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c0c55-deb6-4ab1-9930-274da2afb937 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:26.509391 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c0c55-deb6-4ab1-9930-274da2afb937 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:26.509730 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c0c55-deb6-4ab1-9930-274da2afb937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:26.528457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-772c0c55-deb6-4ab1-9930-274da2afb937 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:10:26.529772 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-772c0c55-deb6-4ab1-9930-274da2afb937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:10:26.538140 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1239/2479] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:10:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:10:26.542997 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65310b-eaa0-4240-b63f-8e5be3e98b50 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:26.553051 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65310b-eaa0-4240-b63f-8e5be3e98b50 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:26.558401 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f888c00f-8223-4346-a1b1-df8a8c301f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:26.559096 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888c00f-8223-4346-a1b1-df8a8c301f0d admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:26.559381 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888c00f-8223-4346-a1b1-df8a8c301f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:26.559658 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888c00f-8223-4346-a1b1-df8a8c301f0d admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:26.560387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f888c00f-8223-4346-a1b1-df8a8c301f0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:10:26.560693 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f888c00f-8223-4346-a1b1-df8a8c301f0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:26.562326 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f888c00f-8223-4346-a1b1-df8a8c301f0d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:26.578027 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1241/2480] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:10:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:10:26.579161 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c0c55-deb6-4ab1-9930-274da2afb937 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:26.590067 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c0c55-deb6-4ab1-9930-274da2afb937 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:26.595149 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca93ba-7cf8-48af-a69a-a74cdf084063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:26.595806 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca93ba-7cf8-48af-a69a-a74cdf084063 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:26.596114 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca93ba-7cf8-48af-a69a-a74cdf084063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:26.596421 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca93ba-7cf8-48af-a69a-a74cdf084063 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:26.597417 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca93ba-7cf8-48af-a69a-a74cdf084063 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:26.599426 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca93ba-7cf8-48af-a69a-a74cdf084063 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:26.628882 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1240/2481] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:10:26 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:10:26.630109 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f888c00f-8223-4346-a1b1-df8a8c301f0d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:26.640561 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f888c00f-8223-4346-a1b1-df8a8c301f0d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:26.695641 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad7093e-30d3-4888-bd5b-767edd4da03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:26.696304 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7093e-30d3-4888-bd5b-767edd4da03c admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:26.696588 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7093e-30d3-4888-bd5b-767edd4da03c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:26.696866 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7093e-30d3-4888-bd5b-767edd4da03c admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:26.697698 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad7093e-30d3-4888-bd5b-767edd4da03c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:26.699531 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad7093e-30d3-4888-bd5b-767edd4da03c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:26.719298 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1242/2482] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:10:26 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:10:26.723722 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca93ba-7cf8-48af-a69a-a74cdf084063 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:26.734622 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca93ba-7cf8-48af-a69a-a74cdf084063 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:26.739244 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2545e-0ac9-46ad-a6e5-3788edc27106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:26.739895 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2545e-0ac9-46ad-a6e5-3788edc27106 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:26.740176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2545e-0ac9-46ad-a6e5-3788edc27106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:26.740471 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2545e-0ac9-46ad-a6e5-3788edc27106 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:26.741325 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e2545e-0ac9-46ad-a6e5-3788edc27106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:26.742970 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e2545e-0ac9-46ad-a6e5-3788edc27106 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:26.753919 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1241/2483] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:10:26.757724 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad7093e-30d3-4888-bd5b-767edd4da03c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:26.767456 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad7093e-30d3-4888-bd5b-767edd4da03c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:26.771956 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0154b537-75cf-4193-95a7-f4cfee6a6d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:26.772680 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154b537-75cf-4193-95a7-f4cfee6a6d16 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:26.772967 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154b537-75cf-4193-95a7-f4cfee6a6d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:26.773240 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154b537-75cf-4193-95a7-f4cfee6a6d16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:26.773988 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0154b537-75cf-4193-95a7-f4cfee6a6d16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:26.775574 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0154b537-75cf-4193-95a7-f4cfee6a6d16 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:26.785324 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1243/2484] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:10:26.789546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2545e-0ac9-46ad-a6e5-3788edc27106 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:26.796640 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:10:26.800998 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2545e-0ac9-46ad-a6e5-3788edc27106 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:26.807854 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97111da0-4178-4733-a8de-1612a93f9b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:26.808697 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97111da0-4178-4733-a8de-1612a93f9b03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:26.809326 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97111da0-4178-4733-a8de-1612a93f9b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:26.809642 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97111da0-4178-4733-a8de-1612a93f9b03 admin admin] PATH_INFO: `/v3/projects/0e53ca846fca4194a95fbdad22d10ed0/users/805889a65fca41b38b334d7a7fdda4ac/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:26.825189 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97111da0-4178-4733-a8de-1612a93f9b03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e53ca846fca4194a95fbdad22d10ed0, user_id=805889a65fca41b38b334d7a7fdda4ac, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:26.827914 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97111da0-4178-4733-a8de-1612a93f9b03 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:26.842737 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1242/2485] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:10:26 2022] PUT /identity/v3/projects/0e53ca846fca4194a95fbdad22d10ed0/users/805889a65fca41b38b334d7a7fdda4ac/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:10:26.848286 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d409f3e-39d7-43f7-99d7-755b2c35077b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:26.848591 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d409f3e-39d7-43f7-99d7-755b2c35077b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:26.848883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d409f3e-39d7-43f7-99d7-755b2c35077b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:26.867791 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6d409f3e-39d7-43f7-99d7-755b2c35077b None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:10:26.869132 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6d409f3e-39d7-43f7-99d7-755b2c35077b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:10:26.929662 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1244/2486] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:10:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:10:26.940063 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97111da0-4178-4733-a8de-1612a93f9b03 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:26.941943 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-97111da0-4178-4733-a8de-1612a93f9b03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:10:26.999769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97111da0-4178-4733-a8de-1612a93f9b03 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:27.003478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eab0ab-dde2-497f-88b6-1565ff87960a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:27.004125 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eab0ab-dde2-497f-88b6-1565ff87960a service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:27.004410 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eab0ab-dde2-497f-88b6-1565ff87960a service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:27.004690 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eab0ab-dde2-497f-88b6-1565ff87960a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:27.021818 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eab0ab-dde2-497f-88b6-1565ff87960a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:27.023288 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eab0ab-dde2-497f-88b6-1565ff87960a service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:27.033433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1243/2487] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:10:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 16:10:32.214727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021a54a-25c8-41bc-9dcc-0ab93e38ec3f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:32.215247 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021a54a-25c8-41bc-9dcc-0ab93e38ec3f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:32.215401 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021a54a-25c8-41bc-9dcc-0ab93e38ec3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:32.253287 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d021a54a-25c8-41bc-9dcc-0ab93e38ec3f None None] MFA Rules not processed for user `805889a65fca41b38b334d7a7fdda4ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:10:32.254678 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d021a54a-25c8-41bc-9dcc-0ab93e38ec3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:10:32.303511 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:10:32.336027 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1245/2488] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:10:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:10:32.351483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eab0ab-dde2-497f-88b6-1565ff87960a service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:32.352988 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-07eab0ab-dde2-497f-88b6-1565ff87960a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:10:32.431730 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eab0ab-dde2-497f-88b6-1565ff87960a service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:32.437336 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a90ef4-0d79-4acd-b9b3-8556cb8dec78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:32.438029 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a90ef4-0d79-4acd-b9b3-8556cb8dec78 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:32.438321 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a90ef4-0d79-4acd-b9b3-8556cb8dec78 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:32.438602 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a90ef4-0d79-4acd-b9b3-8556cb8dec78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:32.448350 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a90ef4-0d79-4acd-b9b3-8556cb8dec78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:32.450031 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a90ef4-0d79-4acd-b9b3-8556cb8dec78 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:32.467997 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1244/2489] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:10:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 16:10:34.293350 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed38d4f-8edf-4f1e-a85f-c5a741b3a4df None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:34.293902 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed38d4f-8edf-4f1e-a85f-c5a741b3a4df None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:34.294002 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed38d4f-8edf-4f1e-a85f-c5a741b3a4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:34.312425 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7ed38d4f-8edf-4f1e-a85f-c5a741b3a4df None None] MFA Rules not processed for user `805889a65fca41b38b334d7a7fdda4ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:10:34.313666 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7ed38d4f-8edf-4f1e-a85f-c5a741b3a4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:10:34.320737 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1246/2490] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:10:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:10:34.627011 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a90ef4-0d79-4acd-b9b3-8556cb8dec78 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:34.636402 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a90ef4-0d79-4acd-b9b3-8556cb8dec78 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:34.640328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae50cb4-e5df-4ff4-b538-07edbbfc0f2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:34.641004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae50cb4-e5df-4ff4-b538-07edbbfc0f2a service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:34.641292 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae50cb4-e5df-4ff4-b538-07edbbfc0f2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:34.641597 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae50cb4-e5df-4ff4-b538-07edbbfc0f2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:34.643374 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5ae50cb4-e5df-4ff4-b538-07edbbfc0f2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:10:34.656740 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae50cb4-e5df-4ff4-b538-07edbbfc0f2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:34.658258 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae50cb4-e5df-4ff4-b538-07edbbfc0f2a service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:34.670198 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1245/2491] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:10:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:10:39.974482 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed38d4f-8edf-4f1e-a85f-c5a741b3a4df None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:39.976113 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7ed38d4f-8edf-4f1e-a85f-c5a741b3a4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:10:40.082071 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed38d4f-8edf-4f1e-a85f-c5a741b3a4df None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:40.088876 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c3e22-9907-46b0-8224-ca5137d874f9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822d9d3521db4a9688f51c6313c7dd74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:40.090999 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c3e22-9907-46b0-8224-ca5137d874f9 service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:40.091511 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c3e22-9907-46b0-8224-ca5137d874f9 service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:40.092017 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c3e22-9907-46b0-8224-ca5137d874f9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:40.094585 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-db2c3e22-9907-46b0-8224-ca5137d874f9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:10:40.113272 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae50cb4-e5df-4ff4-b538-07edbbfc0f2a service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:40.114734 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5ae50cb4-e5df-4ff4-b538-07edbbfc0f2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:10:40.119137 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2c3e22-9907-46b0-8224-ca5137d874f9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:40.121546 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2c3e22-9907-46b0-8224-ca5137d874f9 service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:40.131667 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae50cb4-e5df-4ff4-b538-07edbbfc0f2a service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:40.137928 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ed05f-d2b8-4ae5-8cca-d565de09b919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:40.139259 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ed05f-d2b8-4ae5-8cca-d565de09b919 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:40.139780 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ed05f-d2b8-4ae5-8cca-d565de09b919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:40.140290 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ed05f-d2b8-4ae5-8cca-d565de09b919 admin admin] PATH_INFO: `/v3/users/a50cbf92286c420f9bee6e05c33dc440` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:40.141868 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1247/2492] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:10:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:10:40.150706 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675ed05f-d2b8-4ae5-8cca-d565de09b919 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a50cbf92286c420f9bee6e05c33dc440)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:40.154684 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675ed05f-d2b8-4ae5-8cca-d565de09b919 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:40.262426 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-675ed05f-d2b8-4ae5-8cca-d565de09b919 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a50cbf92286c420f9bee6e05c33dc440'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:10:40.263462 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-675ed05f-d2b8-4ae5-8cca-d565de09b919 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a50cbf92286c420f9bee6e05c33dc440'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:10:40.264755 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-675ed05f-d2b8-4ae5-8cca-d565de09b919 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a50cbf92286c420f9bee6e05c33dc440'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:10:40.268644 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:10:40.282573 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-675ed05f-d2b8-4ae5-8cca-d565de09b919 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a50cbf92286c420f9bee6e05c33dc440'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:10:40.291833 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-675ed05f-d2b8-4ae5-8cca-d565de09b919 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a50cbf92286c420f9bee6e05c33dc440'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:10:40.315515 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1246/2493] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:10:40 2022] DELETE /identity/v3/users/a50cbf92286c420f9bee6e05c33dc440 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:10:40.569004 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c3e22-9907-46b0-8224-ca5137d874f9 service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:40.582165 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c3e22-9907-46b0-8224-ca5137d874f9 service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:40.681223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9672f-ccb4-45df-a950-eb966f3d881b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:40.682668 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9672f-ccb4-45df-a950-eb966f3d881b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:40.682947 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9672f-ccb4-45df-a950-eb966f3d881b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:40.683222 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9672f-ccb4-45df-a950-eb966f3d881b admin admin] PATH_INFO: `/v3/projects/2e279d6a3e6d4ba7b697449111d569c2` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:40.684531 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f9672f-ccb4-45df-a950-eb966f3d881b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e279d6a3e6d4ba7b697449111d569c2)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:40.686517 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f9672f-ccb4-45df-a950-eb966f3d881b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:40.734150 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-73f9672f-ccb4-45df-a950-eb966f3d881b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e279d6a3e6d4ba7b697449111d569c2', 'request_id': 'req-73f9672f-ccb4-45df-a950-eb966f3d881b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:10:40.742071 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-73f9672f-ccb4-45df-a950-eb966f3d881b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e279d6a3e6d4ba7b697449111d569c2', 'request_id': 'req-73f9672f-ccb4-45df-a950-eb966f3d881b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:10:40.751445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-73f9672f-ccb4-45df-a950-eb966f3d881b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e279d6a3e6d4ba7b697449111d569c2', 'request_id': 'req-73f9672f-ccb4-45df-a950-eb966f3d881b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:10:40.756443 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:10:40.765111 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-73f9672f-ccb4-45df-a950-eb966f3d881b admin admin] The token cache is being invalidate because project 2e279d6a3e6d4ba7b697449111d569c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:10:40.765868 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-73f9672f-ccb4-45df-a950-eb966f3d881b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:10:40.766802 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-73f9672f-ccb4-45df-a950-eb966f3d881b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:10:40.768627 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1248/2494] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:10:40 2022] DELETE /identity/v3/projects/2e279d6a3e6d4ba7b697449111d569c2 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:10:40.773599 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c0581-6893-4b28-9cfc-1e2c40595c44 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:40.774171 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c0581-6893-4b28-9cfc-1e2c40595c44 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:40.774171 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c0581-6893-4b28-9cfc-1e2c40595c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:40.805170 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-de1c0581-6893-4b28-9cfc-1e2c40595c44 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:10:40.807360 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-de1c0581-6893-4b28-9cfc-1e2c40595c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:10:40.876142 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1247/2495] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:10:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:10:40.884140 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc796d7-127a-47f4-b898-27d98baa3b50 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:40.884602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc796d7-127a-47f4-b898-27d98baa3b50 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:40.884960 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc796d7-127a-47f4-b898-27d98baa3b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:40.903029 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3bc796d7-127a-47f4-b898-27d98baa3b50 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:10:40.904250 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3bc796d7-127a-47f4-b898-27d98baa3b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:10:40.912312 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1249/2496] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:10:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:10:40.917097 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c0581-6893-4b28-9cfc-1e2c40595c44 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:40.927050 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c0581-6893-4b28-9cfc-1e2c40595c44 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:40.932494 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d78b31-5058-4f9f-83bb-7b5116bd05b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:40.933164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d78b31-5058-4f9f-83bb-7b5116bd05b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:40.933452 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d78b31-5058-4f9f-83bb-7b5116bd05b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:40.933769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d78b31-5058-4f9f-83bb-7b5116bd05b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:40.934628 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d78b31-5058-4f9f-83bb-7b5116bd05b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:10:40.935054 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d78b31-5058-4f9f-83bb-7b5116bd05b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:40.936802 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d78b31-5058-4f9f-83bb-7b5116bd05b8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:40.959609 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1248/2497] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:10:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:10:40.961153 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc796d7-127a-47f4-b898-27d98baa3b50 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:40.972415 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc796d7-127a-47f4-b898-27d98baa3b50 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:40.978877 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3846e7-e7cb-4e54-a451-08744e993dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:40.982775 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3846e7-e7cb-4e54-a451-08744e993dce admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:40.983058 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3846e7-e7cb-4e54-a451-08744e993dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:40.983349 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3846e7-e7cb-4e54-a451-08744e993dce admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:40.984055 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3846e7-e7cb-4e54-a451-08744e993dce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:40.986051 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3846e7-e7cb-4e54-a451-08744e993dce admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:41.016835 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1250/2498] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:10:40 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:10:41.018078 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d78b31-5058-4f9f-83bb-7b5116bd05b8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:41.028553 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d78b31-5058-4f9f-83bb-7b5116bd05b8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:41.085069 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255efe9-a4ac-4134-b3b1-75a127f8d9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:41.085771 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255efe9-a4ac-4134-b3b1-75a127f8d9fe admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:41.086057 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255efe9-a4ac-4134-b3b1-75a127f8d9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:41.086363 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255efe9-a4ac-4134-b3b1-75a127f8d9fe admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:41.087058 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f255efe9-a4ac-4134-b3b1-75a127f8d9fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:41.089009 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f255efe9-a4ac-4134-b3b1-75a127f8d9fe admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:41.110193 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1249/2499] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:10:41 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:10:41.114537 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3846e7-e7cb-4e54-a451-08744e993dce admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:41.131772 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3846e7-e7cb-4e54-a451-08744e993dce admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:41.136507 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c296db79-08dc-4957-bc1c-389c3595c3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:41.137180 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296db79-08dc-4957-bc1c-389c3595c3be admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:41.137473 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296db79-08dc-4957-bc1c-389c3595c3be admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:41.137782 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296db79-08dc-4957-bc1c-389c3595c3be admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:41.138516 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c296db79-08dc-4957-bc1c-389c3595c3be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:41.140125 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c296db79-08dc-4957-bc1c-389c3595c3be admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:41.150683 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1251/2500] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:10:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:10:41.154746 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255efe9-a4ac-4134-b3b1-75a127f8d9fe admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:41.164881 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255efe9-a4ac-4134-b3b1-75a127f8d9fe admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:41.170220 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126af661-ddac-404e-adce-00b749002333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:41.170873 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126af661-ddac-404e-adce-00b749002333 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:41.171164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126af661-ddac-404e-adce-00b749002333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:41.171445 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126af661-ddac-404e-adce-00b749002333 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:41.172175 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126af661-ddac-404e-adce-00b749002333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:41.173802 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126af661-ddac-404e-adce-00b749002333 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:41.184344 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1250/2501] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:10:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:10:41.188454 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c296db79-08dc-4957-bc1c-389c3595c3be admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:41.198304 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c296db79-08dc-4957-bc1c-389c3595c3be admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:41.202846 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68588f67-007e-4028-bbf1-49de08481dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:41.203715 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68588f67-007e-4028-bbf1-49de08481dbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:41.203998 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68588f67-007e-4028-bbf1-49de08481dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:41.204276 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68588f67-007e-4028-bbf1-49de08481dbb admin admin] PATH_INFO: `/v3/projects/f941312b44ec41fcafdaa1c616f00d2a/users/3461f7547dbe44bcb70b37bea6aad3da/roles/54195e660fc349978b39d5794765a095` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:41.221386 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68588f67-007e-4028-bbf1-49de08481dbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f941312b44ec41fcafdaa1c616f00d2a, user_id=3461f7547dbe44bcb70b37bea6aad3da, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:41.224456 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68588f67-007e-4028-bbf1-49de08481dbb admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:41.247931 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1252/2502] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:10:41 2022] PUT /identity/v3/projects/f941312b44ec41fcafdaa1c616f00d2a/users/3461f7547dbe44bcb70b37bea6aad3da/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:10:41.259083 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6de7b-e071-4022-a7cc-2aba5509e7d6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:41.259308 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6de7b-e071-4022-a7cc-2aba5509e7d6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:41.259589 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6de7b-e071-4022-a7cc-2aba5509e7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:41.286721 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-64c6de7b-e071-4022-a7cc-2aba5509e7d6 None None] MFA Rules not processed for user `3461f7547dbe44bcb70b37bea6aad3da`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:10:41.288232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-64c6de7b-e071-4022-a7cc-2aba5509e7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:10:41.309825 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:10:41.357756 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1251/2503] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:10:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:10:41.362206 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68588f67-007e-4028-bbf1-49de08481dbb admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:41.430207 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68588f67-007e-4028-bbf1-49de08481dbb admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:41.434838 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8eebb-d8b1-41fe-b950-6e5e1c0c0d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:41.435489 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8eebb-d8b1-41fe-b950-6e5e1c0c0d14 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:41.435772 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8eebb-d8b1-41fe-b950-6e5e1c0c0d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:41.436046 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8eebb-d8b1-41fe-b950-6e5e1c0c0d14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:41.436729 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae8eebb-d8b1-41fe-b950-6e5e1c0c0d14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:41.438658 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae8eebb-d8b1-41fe-b950-6e5e1c0c0d14 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:41.463114 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1253/2504] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:10:41 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:10:41.469131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c6de7b-e071-4022-a7cc-2aba5509e7d6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:41.482072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c6de7b-e071-4022-a7cc-2aba5509e7d6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:41.539739 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc97697-7c00-464a-bc51-8513ba2eeae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:41.540414 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc97697-7c00-464a-bc51-8513ba2eeae4 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:41.540728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc97697-7c00-464a-bc51-8513ba2eeae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:41.541253 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc97697-7c00-464a-bc51-8513ba2eeae4 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:41.542523 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc97697-7c00-464a-bc51-8513ba2eeae4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:41.544465 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc97697-7c00-464a-bc51-8513ba2eeae4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:41.565778 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1252/2505] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:10:41 2022] POST /identity/v3/users => generated 295 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:10:41.570139 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8eebb-d8b1-41fe-b950-6e5e1c0c0d14 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:41.582089 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8eebb-d8b1-41fe-b950-6e5e1c0c0d14 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:41.586776 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ccd714-23a3-4371-b5b0-f9461c8994d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:41.587446 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ccd714-23a3-4371-b5b0-f9461c8994d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:41.588329 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ccd714-23a3-4371-b5b0-f9461c8994d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:41.588611 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ccd714-23a3-4371-b5b0-f9461c8994d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:41.589336 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ccd714-23a3-4371-b5b0-f9461c8994d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:41.591005 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ccd714-23a3-4371-b5b0-f9461c8994d9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:41.601949 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1254/2506] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:10:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:10:41.606332 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc97697-7c00-464a-bc51-8513ba2eeae4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:41.620809 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc97697-7c00-464a-bc51-8513ba2eeae4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:41.626214 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16fc2a-1d90-4bb1-9813-e753515d8357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:41.627311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16fc2a-1d90-4bb1-9813-e753515d8357 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:41.627664 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16fc2a-1d90-4bb1-9813-e753515d8357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:41.628061 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16fc2a-1d90-4bb1-9813-e753515d8357 admin admin] PATH_INFO: `/v3/projects/c09206b577f44701b2134eb2d1bb932c/users/3e238a5495884fc184a0536d716e056b/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:41.645194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab16fc2a-1d90-4bb1-9813-e753515d8357 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c09206b577f44701b2134eb2d1bb932c, user_id=3e238a5495884fc184a0536d716e056b, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:41.648682 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab16fc2a-1d90-4bb1-9813-e753515d8357 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:41.663933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1253/2507] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:10:41 2022] PUT /identity/v3/projects/c09206b577f44701b2134eb2d1bb932c/users/3e238a5495884fc184a0536d716e056b/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:10:41.667881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ccd714-23a3-4371-b5b0-f9461c8994d9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:41.678224 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ccd714-23a3-4371-b5b0-f9461c8994d9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:41.749557 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4177bb4-b03e-461d-85f2-2fd82ab881aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:41.750263 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4177bb4-b03e-461d-85f2-2fd82ab881aa admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:41.750554 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4177bb4-b03e-461d-85f2-2fd82ab881aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:41.750839 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4177bb4-b03e-461d-85f2-2fd82ab881aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:41.751583 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4177bb4-b03e-461d-85f2-2fd82ab881aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:41.753188 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4177bb4-b03e-461d-85f2-2fd82ab881aa admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:41.764076 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1255/2508] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:10:41 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:10:41.768160 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16fc2a-1d90-4bb1-9813-e753515d8357 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:41.779282 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16fc2a-1d90-4bb1-9813-e753515d8357 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:41.783989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5065de34-fcd2-4951-a0cf-602d70d9dd48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:41.784966 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065de34-fcd2-4951-a0cf-602d70d9dd48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:41.785341 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065de34-fcd2-4951-a0cf-602d70d9dd48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:41.785659 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065de34-fcd2-4951-a0cf-602d70d9dd48 admin admin] PATH_INFO: `/v3/projects/c09206b577f44701b2134eb2d1bb932c/users/3e238a5495884fc184a0536d716e056b/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:41.787826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5065de34-fcd2-4951-a0cf-602d70d9dd48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c09206b577f44701b2134eb2d1bb932c, user_id=3e238a5495884fc184a0536d716e056b, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:41.790671 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5065de34-fcd2-4951-a0cf-602d70d9dd48 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:41.801978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1254/2509] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:10:41 2022] PUT /identity/v3/projects/c09206b577f44701b2134eb2d1bb932c/users/3e238a5495884fc184a0536d716e056b/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:10:41.812128 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c5e21-786a-421a-b987-26da089c1c34 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:41.812426 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c5e21-786a-421a-b987-26da089c1c34 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:41.812727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c5e21-786a-421a-b987-26da089c1c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:41.818780 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:10:41.842777 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-801c5e21-786a-421a-b987-26da089c1c34 None None] MFA Rules not processed for user `3e238a5495884fc184a0536d716e056b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:10:41.844073 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-801c5e21-786a-421a-b987-26da089c1c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:10:41.914233 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1256/2510] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:10:41 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:10:41.925181 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5065de34-fcd2-4951-a0cf-602d70d9dd48 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:41.927007 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5065de34-fcd2-4951-a0cf-602d70d9dd48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:10:41.985584 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5065de34-fcd2-4951-a0cf-602d70d9dd48 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:41.989242 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d26a9-522d-41e8-95fa-9e5ffcfd843e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:41.989924 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d26a9-522d-41e8-95fa-9e5ffcfd843e service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:41.990210 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d26a9-522d-41e8-95fa-9e5ffcfd843e service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:41.990490 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d26a9-522d-41e8-95fa-9e5ffcfd843e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:42.019952 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:10:42.048963 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d26a9-522d-41e8-95fa-9e5ffcfd843e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:42.050368 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d26a9-522d-41e8-95fa-9e5ffcfd843e service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:42.059212 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1255/2511] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:10:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:10:46.689907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c5e21-786a-421a-b987-26da089c1c34 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:10:46.700086 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c5e21-786a-421a-b987-26da089c1c34 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:10:46.704161 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876fc53-1f10-4598-a322-4120a7b9404f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:10:46.704837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876fc53-1f10-4598-a322-4120a7b9404f service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:10:46.705129 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876fc53-1f10-4598-a322-4120a7b9404f service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:10:46.709884 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876fc53-1f10-4598-a322-4120a7b9404f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:10:46.721693 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876fc53-1f10-4598-a322-4120a7b9404f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:10:46.723863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876fc53-1f10-4598-a322-4120a7b9404f service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:10:46.735002 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1257/2512] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:10:46 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:10:52.055108 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=14 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:10:56.730142 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=39 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:10.496507 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d26a9-522d-41e8-95fa-9e5ffcfd843e service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:10.558412 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d26a9-522d-41e8-95fa-9e5ffcfd843e service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:10.562978 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95daee-9b36-4e43-b481-993250b380a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:10.563843 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95daee-9b36-4e43-b481-993250b380a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:10.564128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95daee-9b36-4e43-b481-993250b380a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:10.564406 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95daee-9b36-4e43-b481-993250b380a6 admin admin] PATH_INFO: `/v3/users/3461f7547dbe44bcb70b37bea6aad3da` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:10.565704 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be95daee-9b36-4e43-b481-993250b380a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3461f7547dbe44bcb70b37bea6aad3da)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:10.567636 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be95daee-9b36-4e43-b481-993250b380a6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:10.612742 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-be95daee-9b36-4e43-b481-993250b380a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3461f7547dbe44bcb70b37bea6aad3da'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:10.613394 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-be95daee-9b36-4e43-b481-993250b380a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3461f7547dbe44bcb70b37bea6aad3da'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:10.614035 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-be95daee-9b36-4e43-b481-993250b380a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3461f7547dbe44bcb70b37bea6aad3da'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:10.621351 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-be95daee-9b36-4e43-b481-993250b380a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3461f7547dbe44bcb70b37bea6aad3da'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:10.628706 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-be95daee-9b36-4e43-b481-993250b380a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3461f7547dbe44bcb70b37bea6aad3da'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:10.646310 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1256/2513] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:11:10 2022] DELETE /identity/v3/users/3461f7547dbe44bcb70b37bea6aad3da => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:10.650967 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66524c70-0fa7-4dba-92c5-3b6c6b4a75bd None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:10.651504 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66524c70-0fa7-4dba-92c5-3b6c6b4a75bd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:10.651504 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66524c70-0fa7-4dba-92c5-3b6c6b4a75bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:10.668890 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-66524c70-0fa7-4dba-92c5-3b6c6b4a75bd None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:10.670221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-66524c70-0fa7-4dba-92c5-3b6c6b4a75bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:10.737754 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1258/2514] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:10.748054 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95daee-9b36-4e43-b481-993250b380a6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:10.749458 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-be95daee-9b36-4e43-b481-993250b380a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:10.806075 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95daee-9b36-4e43-b481-993250b380a6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:10.809719 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b38daa8-2032-4cb2-bc8a-b9dbf58d8635 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:10.810364 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b38daa8-2032-4cb2-bc8a-b9dbf58d8635 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:10.810650 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b38daa8-2032-4cb2-bc8a-b9dbf58d8635 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:10.810930 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b38daa8-2032-4cb2-bc8a-b9dbf58d8635 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:10.819890 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b38daa8-2032-4cb2-bc8a-b9dbf58d8635 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:10.821945 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b38daa8-2032-4cb2-bc8a-b9dbf58d8635 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:10.831226 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1257/2515] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:11:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:11:11.108441 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66524c70-0fa7-4dba-92c5-3b6c6b4a75bd None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:11.109913 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-66524c70-0fa7-4dba-92c5-3b6c6b4a75bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:11.123162 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66524c70-0fa7-4dba-92c5-3b6c6b4a75bd None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:11.128003 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c12575-bbfb-4052-a19e-0209725d03e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:11.128872 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c12575-bbfb-4052-a19e-0209725d03e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:11.129158 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c12575-bbfb-4052-a19e-0209725d03e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:11.129441 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c12575-bbfb-4052-a19e-0209725d03e1 admin admin] PATH_INFO: `/v3/projects/f941312b44ec41fcafdaa1c616f00d2a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:11.130679 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c12575-bbfb-4052-a19e-0209725d03e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f941312b44ec41fcafdaa1c616f00d2a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:11.132630 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c12575-bbfb-4052-a19e-0209725d03e1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:11.179338 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e5c12575-bbfb-4052-a19e-0209725d03e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f941312b44ec41fcafdaa1c616f00d2a', 'request_id': 'req-e5c12575-bbfb-4052-a19e-0209725d03e1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:11.186061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e5c12575-bbfb-4052-a19e-0209725d03e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f941312b44ec41fcafdaa1c616f00d2a', 'request_id': 'req-e5c12575-bbfb-4052-a19e-0209725d03e1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:11.193418 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e5c12575-bbfb-4052-a19e-0209725d03e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f941312b44ec41fcafdaa1c616f00d2a', 'request_id': 'req-e5c12575-bbfb-4052-a19e-0209725d03e1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:11.203563 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e5c12575-bbfb-4052-a19e-0209725d03e1 admin admin] The token cache is being invalidate because project f941312b44ec41fcafdaa1c616f00d2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:11:11.203851 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e5c12575-bbfb-4052-a19e-0209725d03e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:11.204469 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e5c12575-bbfb-4052-a19e-0209725d03e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:11.206040 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1259/2516] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:11:11 2022] DELETE /identity/v3/projects/f941312b44ec41fcafdaa1c616f00d2a => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:11.210058 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b38daa8-2032-4cb2-bc8a-b9dbf58d8635 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:11.220351 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b38daa8-2032-4cb2-bc8a-b9dbf58d8635 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:11.240433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:11.281025 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae3c472-c5e0-4f1a-9a51-5296290cfc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:11.281920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae3c472-c5e0-4f1a-9a51-5296290cfc9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:11.282206 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae3c472-c5e0-4f1a-9a51-5296290cfc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:11.282481 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae3c472-c5e0-4f1a-9a51-5296290cfc9c admin admin] PATH_INFO: `/v3/users/3e238a5495884fc184a0536d716e056b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:11.283661 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae3c472-c5e0-4f1a-9a51-5296290cfc9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e238a5495884fc184a0536d716e056b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:11.285629 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae3c472-c5e0-4f1a-9a51-5296290cfc9c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:11.332274 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0ae3c472-c5e0-4f1a-9a51-5296290cfc9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e238a5495884fc184a0536d716e056b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:11.332923 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0ae3c472-c5e0-4f1a-9a51-5296290cfc9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e238a5495884fc184a0536d716e056b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:11.333515 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0ae3c472-c5e0-4f1a-9a51-5296290cfc9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e238a5495884fc184a0536d716e056b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:11.341042 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0ae3c472-c5e0-4f1a-9a51-5296290cfc9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e238a5495884fc184a0536d716e056b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:11.348649 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0ae3c472-c5e0-4f1a-9a51-5296290cfc9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e238a5495884fc184a0536d716e056b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:11.366178 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1258/2517] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:11:11 2022] DELETE /identity/v3/users/3e238a5495884fc184a0536d716e056b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:11.733018 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c12575-bbfb-4052-a19e-0209725d03e1 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:11.734490 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e5c12575-bbfb-4052-a19e-0209725d03e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:11.748978 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c12575-bbfb-4052-a19e-0209725d03e1 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:11.807832 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92713a1-10ab-4530-b3d6-eac2bf61f96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:11.808729 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92713a1-10ab-4530-b3d6-eac2bf61f96a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:11.809018 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92713a1-10ab-4530-b3d6-eac2bf61f96a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:11.809299 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92713a1-10ab-4530-b3d6-eac2bf61f96a admin admin] PATH_INFO: `/v3/projects/c09206b577f44701b2134eb2d1bb932c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:11.810522 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92713a1-10ab-4530-b3d6-eac2bf61f96a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c09206b577f44701b2134eb2d1bb932c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:11.812456 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92713a1-10ab-4530-b3d6-eac2bf61f96a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:11.842666 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:11.860559 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b92713a1-10ab-4530-b3d6-eac2bf61f96a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c09206b577f44701b2134eb2d1bb932c', 'request_id': 'req-b92713a1-10ab-4530-b3d6-eac2bf61f96a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:11.866652 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b92713a1-10ab-4530-b3d6-eac2bf61f96a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c09206b577f44701b2134eb2d1bb932c', 'request_id': 'req-b92713a1-10ab-4530-b3d6-eac2bf61f96a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:11.873834 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b92713a1-10ab-4530-b3d6-eac2bf61f96a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c09206b577f44701b2134eb2d1bb932c', 'request_id': 'req-b92713a1-10ab-4530-b3d6-eac2bf61f96a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:11.884501 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b92713a1-10ab-4530-b3d6-eac2bf61f96a admin admin] The token cache is being invalidate because project c09206b577f44701b2134eb2d1bb932c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:11:11.884792 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b92713a1-10ab-4530-b3d6-eac2bf61f96a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:11.885422 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b92713a1-10ab-4530-b3d6-eac2bf61f96a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:11.886984 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1260/2518] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:11:11 2022] DELETE /identity/v3/projects/c09206b577f44701b2134eb2d1bb932c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:11.891810 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0429308-b246-49e2-9b9e-663117448617 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:11.892277 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0429308-b246-49e2-9b9e-663117448617 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:11.892391 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0429308-b246-49e2-9b9e-663117448617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:11.910226 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f0429308-b246-49e2-9b9e-663117448617 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:11.911400 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f0429308-b246-49e2-9b9e-663117448617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:11.970694 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1259/2519] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:11.979481 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91564ad1-eca5-43f4-8ae2-984ffb072d73 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:11.979824 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91564ad1-eca5-43f4-8ae2-984ffb072d73 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:11.980110 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91564ad1-eca5-43f4-8ae2-984ffb072d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:11.998878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-91564ad1-eca5-43f4-8ae2-984ffb072d73 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:12.000078 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-91564ad1-eca5-43f4-8ae2-984ffb072d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:12.007995 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1261/2520] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:12.012693 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0429308-b246-49e2-9b9e-663117448617 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:12.022321 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0429308-b246-49e2-9b9e-663117448617 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:12.027354 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e250503-81b7-465e-a8e4-dd9c369b19b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:12.027999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e250503-81b7-465e-a8e4-dd9c369b19b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:12.028281 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e250503-81b7-465e-a8e4-dd9c369b19b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:12.028557 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e250503-81b7-465e-a8e4-dd9c369b19b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:12.029281 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e250503-81b7-465e-a8e4-dd9c369b19b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:11:12.029622 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e250503-81b7-465e-a8e4-dd9c369b19b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:12.031236 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e250503-81b7-465e-a8e4-dd9c369b19b0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:12.043471 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1260/2521] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:11:12.047855 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91564ad1-eca5-43f4-8ae2-984ffb072d73 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:12.057015 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91564ad1-eca5-43f4-8ae2-984ffb072d73 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:12.061631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe6cae-a8a8-4230-9478-ba527e0b0eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:12.062279 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe6cae-a8a8-4230-9478-ba527e0b0eea admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:12.062563 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe6cae-a8a8-4230-9478-ba527e0b0eea admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:12.062840 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe6cae-a8a8-4230-9478-ba527e0b0eea admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:12.063540 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe6cae-a8a8-4230-9478-ba527e0b0eea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:12.065437 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe6cae-a8a8-4230-9478-ba527e0b0eea admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:12.092638 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1262/2522] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:11:12 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:11:12.096768 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e250503-81b7-465e-a8e4-dd9c369b19b0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:12.107540 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e250503-81b7-465e-a8e4-dd9c369b19b0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:12.132850 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:12.168295 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac808992-234c-47f8-bfca-9056dcc00903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:12.169546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac808992-234c-47f8-bfca-9056dcc00903 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:12.169881 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac808992-234c-47f8-bfca-9056dcc00903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:12.170160 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac808992-234c-47f8-bfca-9056dcc00903 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:12.170865 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac808992-234c-47f8-bfca-9056dcc00903 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:12.172860 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac808992-234c-47f8-bfca-9056dcc00903 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:12.192405 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1261/2523] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:11:12 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:11:12.196505 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe6cae-a8a8-4230-9478-ba527e0b0eea admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:12.206917 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe6cae-a8a8-4230-9478-ba527e0b0eea admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:12.212728 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f0a0b-01a9-4c07-9044-d9a0612ca366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:12.213478 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f0a0b-01a9-4c07-9044-d9a0612ca366 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:12.213714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f0a0b-01a9-4c07-9044-d9a0612ca366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:12.214052 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f0a0b-01a9-4c07-9044-d9a0612ca366 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:12.214728 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f0a0b-01a9-4c07-9044-d9a0612ca366 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:12.216341 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f0a0b-01a9-4c07-9044-d9a0612ca366 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:12.227483 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1263/2524] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:11:12.231534 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac808992-234c-47f8-bfca-9056dcc00903 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:12.245944 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac808992-234c-47f8-bfca-9056dcc00903 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:12.250864 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88119532-3880-40d7-9027-3bb8e3a8c22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:12.251531 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88119532-3880-40d7-9027-3bb8e3a8c22b admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:12.251815 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88119532-3880-40d7-9027-3bb8e3a8c22b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:12.252089 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88119532-3880-40d7-9027-3bb8e3a8c22b admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:12.252818 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88119532-3880-40d7-9027-3bb8e3a8c22b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:12.254542 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88119532-3880-40d7-9027-3bb8e3a8c22b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:12.265478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1262/2525] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:11:12.269425 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f0a0b-01a9-4c07-9044-d9a0612ca366 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:12.280081 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f0a0b-01a9-4c07-9044-d9a0612ca366 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:12.284759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66eea6-531a-4ecb-a8b6-4bb3ccf816bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:12.285697 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66eea6-531a-4ecb-a8b6-4bb3ccf816bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:12.285984 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66eea6-531a-4ecb-a8b6-4bb3ccf816bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:12.286263 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66eea6-531a-4ecb-a8b6-4bb3ccf816bd admin admin] PATH_INFO: `/v3/projects/e347e5676f764eeea888f578a0a08289/users/ca249d9d1b154dfbba22f2b12f158aaf/roles/54195e660fc349978b39d5794765a095` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:12.300747 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c66eea6-531a-4ecb-a8b6-4bb3ccf816bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e347e5676f764eeea888f578a0a08289, user_id=ca249d9d1b154dfbba22f2b12f158aaf, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:12.303430 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c66eea6-531a-4ecb-a8b6-4bb3ccf816bd admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:12.317536 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1264/2526] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:11:12 2022] PUT /identity/v3/projects/e347e5676f764eeea888f578a0a08289/users/ca249d9d1b154dfbba22f2b12f158aaf/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:12.327549 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac2cfb-1779-4506-9e8f-9361999a8a13 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:12.327836 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac2cfb-1779-4506-9e8f-9361999a8a13 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:12.328111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac2cfb-1779-4506-9e8f-9361999a8a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:12.355092 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3aac2cfb-1779-4506-9e8f-9361999a8a13 None None] MFA Rules not processed for user `ca249d9d1b154dfbba22f2b12f158aaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:12.356264 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3aac2cfb-1779-4506-9e8f-9361999a8a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:12.427358 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1263/2527] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:12.432727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66eea6-531a-4ecb-a8b6-4bb3ccf816bd admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:12.504306 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66eea6-531a-4ecb-a8b6-4bb3ccf816bd admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:12.509249 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6630546-9fe4-43d6-ab4a-a982ba57a4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:12.510026 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6630546-9fe4-43d6-ab4a-a982ba57a4b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:12.510316 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6630546-9fe4-43d6-ab4a-a982ba57a4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:12.510598 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6630546-9fe4-43d6-ab4a-a982ba57a4b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:12.511298 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6630546-9fe4-43d6-ab4a-a982ba57a4b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:12.513232 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6630546-9fe4-43d6-ab4a-a982ba57a4b8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:12.539074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1265/2528] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:11:12 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:11:12.543261 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac2cfb-1779-4506-9e8f-9361999a8a13 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:12.554259 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac2cfb-1779-4506-9e8f-9361999a8a13 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:12.613096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d8a61-73fb-44ab-849f-a6f957b74207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:12.613827 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d8a61-73fb-44ab-849f-a6f957b74207 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:12.614117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d8a61-73fb-44ab-849f-a6f957b74207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:12.614393 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d8a61-73fb-44ab-849f-a6f957b74207 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:12.615097 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d8a61-73fb-44ab-849f-a6f957b74207 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:12.617076 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d8a61-73fb-44ab-849f-a6f957b74207 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:12.640307 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1264/2529] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:11:12 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:11:12.641620 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6630546-9fe4-43d6-ab4a-a982ba57a4b8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:12.652966 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6630546-9fe4-43d6-ab4a-a982ba57a4b8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:12.659722 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f5dcd-61c7-4e95-aed9-c8b693470882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:12.660059 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f5dcd-61c7-4e95-aed9-c8b693470882 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:12.660731 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f5dcd-61c7-4e95-aed9-c8b693470882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:12.660731 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f5dcd-61c7-4e95-aed9-c8b693470882 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:12.661356 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1f5dcd-61c7-4e95-aed9-c8b693470882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:12.663165 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1f5dcd-61c7-4e95-aed9-c8b693470882 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:12.675508 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1266/2530] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:11:12.679971 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d8a61-73fb-44ab-849f-a6f957b74207 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:12.690619 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d8a61-73fb-44ab-849f-a6f957b74207 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:12.695212 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb63964-8a3c-4db2-bf69-dc449b89e8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:12.696232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb63964-8a3c-4db2-bf69-dc449b89e8b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:12.696510 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb63964-8a3c-4db2-bf69-dc449b89e8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:12.696788 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb63964-8a3c-4db2-bf69-dc449b89e8b2 admin admin] PATH_INFO: `/v3/projects/2073b2e3f74d41919b1405e9651df439/users/8fcd4760ed684e59adc1b2fd93d3a0b8/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:12.712807 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb63964-8a3c-4db2-bf69-dc449b89e8b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2073b2e3f74d41919b1405e9651df439, user_id=8fcd4760ed684e59adc1b2fd93d3a0b8, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:12.715507 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb63964-8a3c-4db2-bf69-dc449b89e8b2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:12.731166 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1265/2531] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:11:12 2022] PUT /identity/v3/projects/2073b2e3f74d41919b1405e9651df439/users/8fcd4760ed684e59adc1b2fd93d3a0b8/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:12.735058 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f5dcd-61c7-4e95-aed9-c8b693470882 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:12.748712 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f5dcd-61c7-4e95-aed9-c8b693470882 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:12.773692 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:12.816280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa282b-0d51-44cf-b013-333b72fd0880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:12.816954 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa282b-0d51-44cf-b013-333b72fd0880 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:12.817241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa282b-0d51-44cf-b013-333b72fd0880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:12.817521 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa282b-0d51-44cf-b013-333b72fd0880 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:12.818288 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daa282b-0d51-44cf-b013-333b72fd0880 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:12.819898 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daa282b-0d51-44cf-b013-333b72fd0880 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:12.829904 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1267/2532] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 16:11:12.833852 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb63964-8a3c-4db2-bf69-dc449b89e8b2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:12.839859 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:12.846525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb63964-8a3c-4db2-bf69-dc449b89e8b2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:12.853105 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea34aff-c00c-4806-82e0-c6701495e3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:12.854255 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea34aff-c00c-4806-82e0-c6701495e3a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:12.854669 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea34aff-c00c-4806-82e0-c6701495e3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:12.855046 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea34aff-c00c-4806-82e0-c6701495e3a3 admin admin] PATH_INFO: `/v3/projects/2073b2e3f74d41919b1405e9651df439/users/8fcd4760ed684e59adc1b2fd93d3a0b8/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:12.857309 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea34aff-c00c-4806-82e0-c6701495e3a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2073b2e3f74d41919b1405e9651df439, user_id=8fcd4760ed684e59adc1b2fd93d3a0b8, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:12.860133 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea34aff-c00c-4806-82e0-c6701495e3a3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:12.871700 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1266/2533] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:11:12 2022] PUT /identity/v3/projects/2073b2e3f74d41919b1405e9651df439/users/8fcd4760ed684e59adc1b2fd93d3a0b8/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:12.881609 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a43ce3-fba7-44fb-a8ad-a9570e39406c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:12.881902 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a43ce3-fba7-44fb-a8ad-a9570e39406c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:12.882189 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a43ce3-fba7-44fb-a8ad-a9570e39406c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:12.908535 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-93a43ce3-fba7-44fb-a8ad-a9570e39406c None None] MFA Rules not processed for user `8fcd4760ed684e59adc1b2fd93d3a0b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:12.909787 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-93a43ce3-fba7-44fb-a8ad-a9570e39406c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:13.001458 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1268/2534] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:12 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:13.011366 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea34aff-c00c-4806-82e0-c6701495e3a3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:13.013170 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5ea34aff-c00c-4806-82e0-c6701495e3a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:13.111820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea34aff-c00c-4806-82e0-c6701495e3a3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:13.117303 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83da66-34aa-4dd6-b59a-fb26e6344a67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:13.118974 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83da66-34aa-4dd6-b59a-fb26e6344a67 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:13.119273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83da66-34aa-4dd6-b59a-fb26e6344a67 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:13.120041 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83da66-34aa-4dd6-b59a-fb26e6344a67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:13.131984 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b83da66-34aa-4dd6-b59a-fb26e6344a67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:13.134164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b83da66-34aa-4dd6-b59a-fb26e6344a67 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:13.158490 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1267/2535] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:11:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:11:13.220797 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a43ce3-fba7-44fb-a8ad-a9570e39406c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:13.311490 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a43ce3-fba7-44fb-a8ad-a9570e39406c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:13.316682 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15f0fc-ea72-46c9-8273-b23c5fca6360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:13.317700 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15f0fc-ea72-46c9-8273-b23c5fca6360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:13.318058 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15f0fc-ea72-46c9-8273-b23c5fca6360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:13.318998 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15f0fc-ea72-46c9-8273-b23c5fca6360 admin admin] PATH_INFO: `/v3/users/ca249d9d1b154dfbba22f2b12f158aaf` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:13.320359 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15f0fc-ea72-46c9-8273-b23c5fca6360 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca249d9d1b154dfbba22f2b12f158aaf)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:13.323414 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15f0fc-ea72-46c9-8273-b23c5fca6360 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:13.390369 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4a15f0fc-ea72-46c9-8273-b23c5fca6360 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca249d9d1b154dfbba22f2b12f158aaf'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:13.391423 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4a15f0fc-ea72-46c9-8273-b23c5fca6360 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca249d9d1b154dfbba22f2b12f158aaf'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:13.392945 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4a15f0fc-ea72-46c9-8273-b23c5fca6360 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca249d9d1b154dfbba22f2b12f158aaf'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:13.394659 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:13.403049 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4a15f0fc-ea72-46c9-8273-b23c5fca6360 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca249d9d1b154dfbba22f2b12f158aaf'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:13.410981 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4a15f0fc-ea72-46c9-8273-b23c5fca6360 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca249d9d1b154dfbba22f2b12f158aaf'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:13.431153 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1269/2536] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:11:13 2022] DELETE /identity/v3/users/ca249d9d1b154dfbba22f2b12f158aaf => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:13.435802 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4a641-98f0-4d19-97d1-11cd4080a2d4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:13.436023 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4a641-98f0-4d19-97d1-11cd4080a2d4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:13.436304 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4a641-98f0-4d19-97d1-11cd4080a2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:13.457134 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-03f4a641-98f0-4d19-97d1-11cd4080a2d4 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:13.458617 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-03f4a641-98f0-4d19-97d1-11cd4080a2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:13.525556 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1268/2537] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:13.535420 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15f0fc-ea72-46c9-8273-b23c5fca6360 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:13.536983 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4a15f0fc-ea72-46c9-8273-b23c5fca6360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:13.595713 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15f0fc-ea72-46c9-8273-b23c5fca6360 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:13.599615 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff8e00-044c-49b8-8e6c-ea2d3ed023e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:13.600265 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff8e00-044c-49b8-8e6c-ea2d3ed023e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:13.600553 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff8e00-044c-49b8-8e6c-ea2d3ed023e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:13.600832 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff8e00-044c-49b8-8e6c-ea2d3ed023e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:13.610563 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff8e00-044c-49b8-8e6c-ea2d3ed023e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:13.612001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff8e00-044c-49b8-8e6c-ea2d3ed023e6 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:13.622091 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1270/2538] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:11:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:11:13.920421 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4a641-98f0-4d19-97d1-11cd4080a2d4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:13.925485 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-03f4a641-98f0-4d19-97d1-11cd4080a2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:13.941132 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4a641-98f0-4d19-97d1-11cd4080a2d4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:13.951398 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f1eb99-fa62-4677-99b6-b044056bc7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:13.953116 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1eb99-fa62-4677-99b6-b044056bc7d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:13.953684 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1eb99-fa62-4677-99b6-b044056bc7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:13.953834 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1eb99-fa62-4677-99b6-b044056bc7d9 admin admin] PATH_INFO: `/v3/projects/e347e5676f764eeea888f578a0a08289` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:13.956138 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f1eb99-fa62-4677-99b6-b044056bc7d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e347e5676f764eeea888f578a0a08289)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:13.958307 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f1eb99-fa62-4677-99b6-b044056bc7d9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:14.022452 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f2f1eb99-fa62-4677-99b6-b044056bc7d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e347e5676f764eeea888f578a0a08289', 'request_id': 'req-f2f1eb99-fa62-4677-99b6-b044056bc7d9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:14.031913 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f2f1eb99-fa62-4677-99b6-b044056bc7d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e347e5676f764eeea888f578a0a08289', 'request_id': 'req-f2f1eb99-fa62-4677-99b6-b044056bc7d9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:14.040196 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f2f1eb99-fa62-4677-99b6-b044056bc7d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e347e5676f764eeea888f578a0a08289', 'request_id': 'req-f2f1eb99-fa62-4677-99b6-b044056bc7d9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:14.053829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f2f1eb99-fa62-4677-99b6-b044056bc7d9 admin admin] The token cache is being invalidate because project e347e5676f764eeea888f578a0a08289 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:11:14.056912 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f2f1eb99-fa62-4677-99b6-b044056bc7d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:14.058352 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f2f1eb99-fa62-4677-99b6-b044056bc7d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:14.060389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1269/2539] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:11:13 2022] DELETE /identity/v3/projects/e347e5676f764eeea888f578a0a08289 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:14.064169 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff8e00-044c-49b8-8e6c-ea2d3ed023e6 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:14.076833 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff8e00-044c-49b8-8e6c-ea2d3ed023e6 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:14.142039 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4b903-3229-4140-b6e7-9c8b60ec59ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:14.143011 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4b903-3229-4140-b6e7-9c8b60ec59ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:14.143301 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4b903-3229-4140-b6e7-9c8b60ec59ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:14.143584 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4b903-3229-4140-b6e7-9c8b60ec59ee admin admin] PATH_INFO: `/v3/users/8fcd4760ed684e59adc1b2fd93d3a0b8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:14.144846 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4b903-3229-4140-b6e7-9c8b60ec59ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fcd4760ed684e59adc1b2fd93d3a0b8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:14.147050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4b903-3229-4140-b6e7-9c8b60ec59ee admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:14.213538 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bde4b903-3229-4140-b6e7-9c8b60ec59ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fcd4760ed684e59adc1b2fd93d3a0b8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:14.214350 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bde4b903-3229-4140-b6e7-9c8b60ec59ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fcd4760ed684e59adc1b2fd93d3a0b8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:14.215138 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bde4b903-3229-4140-b6e7-9c8b60ec59ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fcd4760ed684e59adc1b2fd93d3a0b8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:14.223864 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bde4b903-3229-4140-b6e7-9c8b60ec59ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fcd4760ed684e59adc1b2fd93d3a0b8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:14.233769 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bde4b903-3229-4140-b6e7-9c8b60ec59ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fcd4760ed684e59adc1b2fd93d3a0b8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:14.243640 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:14.261048 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1271/2540] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:11:14 2022] DELETE /identity/v3/users/8fcd4760ed684e59adc1b2fd93d3a0b8 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:14.703206 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f1eb99-fa62-4677-99b6-b044056bc7d9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:14.705221 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f2f1eb99-fa62-4677-99b6-b044056bc7d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:14.722833 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f1eb99-fa62-4677-99b6-b044056bc7d9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:14.745295 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:14.799069 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee793a-570a-4c66-ba19-ed58e3bfdec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:14.800156 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee793a-570a-4c66-ba19-ed58e3bfdec0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:14.800381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee793a-570a-4c66-ba19-ed58e3bfdec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:14.800490 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee793a-570a-4c66-ba19-ed58e3bfdec0 admin admin] PATH_INFO: `/v3/projects/2073b2e3f74d41919b1405e9651df439` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:14.801718 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ee793a-570a-4c66-ba19-ed58e3bfdec0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2073b2e3f74d41919b1405e9651df439)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:14.803877 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ee793a-570a-4c66-ba19-ed58e3bfdec0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:14.853090 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-04ee793a-570a-4c66-ba19-ed58e3bfdec0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2073b2e3f74d41919b1405e9651df439', 'request_id': 'req-04ee793a-570a-4c66-ba19-ed58e3bfdec0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:14.860412 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-04ee793a-570a-4c66-ba19-ed58e3bfdec0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2073b2e3f74d41919b1405e9651df439', 'request_id': 'req-04ee793a-570a-4c66-ba19-ed58e3bfdec0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:14.865975 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-04ee793a-570a-4c66-ba19-ed58e3bfdec0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2073b2e3f74d41919b1405e9651df439', 'request_id': 'req-04ee793a-570a-4c66-ba19-ed58e3bfdec0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:14.877319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-04ee793a-570a-4c66-ba19-ed58e3bfdec0 admin admin] The token cache is being invalidate because project 2073b2e3f74d41919b1405e9651df439 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:11:14.877640 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-04ee793a-570a-4c66-ba19-ed58e3bfdec0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:14.878253 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-04ee793a-570a-4c66-ba19-ed58e3bfdec0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:14.879890 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1270/2541] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:11:14 2022] DELETE /identity/v3/projects/2073b2e3f74d41919b1405e9651df439 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:14.885297 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf4bff-dc52-4229-91d1-5637c00559f4 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:14.885619 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf4bff-dc52-4229-91d1-5637c00559f4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:14.885902 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf4bff-dc52-4229-91d1-5637c00559f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:14.904204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8fcf4bff-dc52-4229-91d1-5637c00559f4 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:14.905443 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8fcf4bff-dc52-4229-91d1-5637c00559f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:14.965326 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1272/2542] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:14.974311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d5a0d-e127-4f80-b512-9ebf0629986f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:14.974595 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d5a0d-e127-4f80-b512-9ebf0629986f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:14.974873 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d5a0d-e127-4f80-b512-9ebf0629986f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:14.994543 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e98d5a0d-e127-4f80-b512-9ebf0629986f None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:14.995714 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e98d5a0d-e127-4f80-b512-9ebf0629986f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:15.004423 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1271/2543] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:15.009380 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf4bff-dc52-4229-91d1-5637c00559f4 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:15.019619 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf4bff-dc52-4229-91d1-5637c00559f4 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:15.024292 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735b3b5-5b65-461e-9ff0-4b31a6789412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:15.024968 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735b3b5-5b65-461e-9ff0-4b31a6789412 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:15.025258 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735b3b5-5b65-461e-9ff0-4b31a6789412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:15.025543 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735b3b5-5b65-461e-9ff0-4b31a6789412 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:15.026308 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7735b3b5-5b65-461e-9ff0-4b31a6789412 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:11:15.026618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7735b3b5-5b65-461e-9ff0-4b31a6789412 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:15.028231 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7735b3b5-5b65-461e-9ff0-4b31a6789412 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:15.040847 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1273/2544] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:11:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 16:11:15.045644 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d5a0d-e127-4f80-b512-9ebf0629986f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:15.055976 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d5a0d-e127-4f80-b512-9ebf0629986f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:15.061168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba84fe1-d3a4-40e5-ac33-4c2d8d387723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:15.061942 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba84fe1-d3a4-40e5-ac33-4c2d8d387723 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:15.062249 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba84fe1-d3a4-40e5-ac33-4c2d8d387723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:15.062561 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba84fe1-d3a4-40e5-ac33-4c2d8d387723 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:15.063268 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba84fe1-d3a4-40e5-ac33-4c2d8d387723 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:15.065335 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba84fe1-d3a4-40e5-ac33-4c2d8d387723 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:15.092219 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1272/2545] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:11:15 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:11:15.096236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735b3b5-5b65-461e-9ff0-4b31a6789412 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:15.106743 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735b3b5-5b65-461e-9ff0-4b31a6789412 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:15.161795 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd80b7-a7e5-4aef-8158-aeab7ab986d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:15.162464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd80b7-a7e5-4aef-8158-aeab7ab986d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:15.162752 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd80b7-a7e5-4aef-8158-aeab7ab986d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:15.163034 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd80b7-a7e5-4aef-8158-aeab7ab986d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:15.163730 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd80b7-a7e5-4aef-8158-aeab7ab986d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:15.165695 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd80b7-a7e5-4aef-8158-aeab7ab986d8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:15.185408 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1274/2546] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:11:15 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:11:15.189657 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba84fe1-d3a4-40e5-ac33-4c2d8d387723 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:15.200080 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba84fe1-d3a4-40e5-ac33-4c2d8d387723 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:15.204682 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f7dad-7378-4707-bfec-675b9c16551e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:15.205339 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f7dad-7378-4707-bfec-675b9c16551e admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:15.205664 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f7dad-7378-4707-bfec-675b9c16551e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:15.205954 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f7dad-7378-4707-bfec-675b9c16551e admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:15.206682 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73f7dad-7378-4707-bfec-675b9c16551e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:15.208321 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73f7dad-7378-4707-bfec-675b9c16551e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:15.219189 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1273/2547] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:11:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:11:15.223211 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd80b7-a7e5-4aef-8158-aeab7ab986d8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:15.234160 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd80b7-a7e5-4aef-8158-aeab7ab986d8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:15.239511 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e566e6-72f6-4e8a-8591-0ab0381e0b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:15.240206 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e566e6-72f6-4e8a-8591-0ab0381e0b71 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:15.240493 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e566e6-72f6-4e8a-8591-0ab0381e0b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:15.240771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e566e6-72f6-4e8a-8591-0ab0381e0b71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:15.241505 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e566e6-72f6-4e8a-8591-0ab0381e0b71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:15.243168 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e566e6-72f6-4e8a-8591-0ab0381e0b71 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:15.253453 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1275/2548] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:11:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:11:15.257782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f7dad-7378-4707-bfec-675b9c16551e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:15.267706 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f7dad-7378-4707-bfec-675b9c16551e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:15.272289 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814002a-f6b2-4e08-ac6e-1b3a18104aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:15.273155 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814002a-f6b2-4e08-ac6e-1b3a18104aec admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:15.273437 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814002a-f6b2-4e08-ac6e-1b3a18104aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:15.274244 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814002a-f6b2-4e08-ac6e-1b3a18104aec admin admin] PATH_INFO: `/v3/projects/16826e7654104049808d2cf7fe78fbda/users/479293e2b0894a5e8dd99f56a811a14f/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:15.289248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8814002a-f6b2-4e08-ac6e-1b3a18104aec admin admin] RBAC: Authorizing `identity:create_grant(project_id=16826e7654104049808d2cf7fe78fbda, user_id=479293e2b0894a5e8dd99f56a811a14f, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:15.292570 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8814002a-f6b2-4e08-ac6e-1b3a18104aec admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:15.304994 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1274/2549] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:11:15 2022] PUT /identity/v3/projects/16826e7654104049808d2cf7fe78fbda/users/479293e2b0894a5e8dd99f56a811a14f/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:11:15.314914 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a1e31-2605-49ae-bbd2-09e9c0bc3cf0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:15.315202 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a1e31-2605-49ae-bbd2-09e9c0bc3cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:15.315482 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a1e31-2605-49ae-bbd2-09e9c0bc3cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:15.341984 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-563a1e31-2605-49ae-bbd2-09e9c0bc3cf0 None None] MFA Rules not processed for user `479293e2b0894a5e8dd99f56a811a14f`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:15.343191 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-563a1e31-2605-49ae-bbd2-09e9c0bc3cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:15.384828 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:15.408540 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1276/2550] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:15.419612 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814002a-f6b2-4e08-ac6e-1b3a18104aec admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:15.421395 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8814002a-f6b2-4e08-ac6e-1b3a18104aec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:15.468198 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:15.483613 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814002a-f6b2-4e08-ac6e-1b3a18104aec admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:15.488013 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d0742-e939-4285-be00-8984775c8011 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:15.488626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d0742-e939-4285-be00-8984775c8011 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:15.488912 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d0742-e939-4285-be00-8984775c8011 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:15.489190 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d0742-e939-4285-be00-8984775c8011 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:15.499075 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5d0742-e939-4285-be00-8984775c8011 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:15.500541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5d0742-e939-4285-be00-8984775c8011 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:15.509452 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1275/2551] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:11:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:11:17.143871 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a1e31-2605-49ae-bbd2-09e9c0bc3cf0 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:17.145295 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-563a1e31-2605-49ae-bbd2-09e9c0bc3cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:17.202428 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a1e31-2605-49ae-bbd2-09e9c0bc3cf0 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:17.205970 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeae286-a7d2-47af-b2c8-b37509c250c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:17.206612 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeae286-a7d2-47af-b2c8-b37509c250c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:17.206900 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeae286-a7d2-47af-b2c8-b37509c250c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:17.207180 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeae286-a7d2-47af-b2c8-b37509c250c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:17.208931 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bbeae286-a7d2-47af-b2c8-b37509c250c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:17.271313 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeae286-a7d2-47af-b2c8-b37509c250c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:17.272753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeae286-a7d2-47af-b2c8-b37509c250c7 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:17.283034 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1277/2552] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:11:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:11:20.139540 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d0742-e939-4285-be00-8984775c8011 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:20.141016 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ad5d0742-e939-4285-be00-8984775c8011 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:20.162447 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d0742-e939-4285-be00-8984775c8011 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:20.168199 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1603831-54be-4dc3-aff7-5ea17ae99ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:20.169072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1603831-54be-4dc3-aff7-5ea17ae99ca7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:20.169360 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1603831-54be-4dc3-aff7-5ea17ae99ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:20.169684 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1603831-54be-4dc3-aff7-5ea17ae99ca7 admin admin] PATH_INFO: `/v3/users/3f984e42c1d3464f94ea9693a9eeb55a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:20.170913 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1603831-54be-4dc3-aff7-5ea17ae99ca7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f984e42c1d3464f94ea9693a9eeb55a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:20.172845 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1603831-54be-4dc3-aff7-5ea17ae99ca7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:20.225746 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f1603831-54be-4dc3-aff7-5ea17ae99ca7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f984e42c1d3464f94ea9693a9eeb55a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:20.226498 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f1603831-54be-4dc3-aff7-5ea17ae99ca7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f984e42c1d3464f94ea9693a9eeb55a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:20.227678 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f1603831-54be-4dc3-aff7-5ea17ae99ca7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f984e42c1d3464f94ea9693a9eeb55a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:20.240234 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f1603831-54be-4dc3-aff7-5ea17ae99ca7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f984e42c1d3464f94ea9693a9eeb55a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:20.248054 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f1603831-54be-4dc3-aff7-5ea17ae99ca7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f984e42c1d3464f94ea9693a9eeb55a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:20.280813 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1276/2553] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:11:20 2022] DELETE /identity/v3/users/3f984e42c1d3464f94ea9693a9eeb55a => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:20.456150 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeae286-a7d2-47af-b2c8-b37509c250c7 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:20.466793 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeae286-a7d2-47af-b2c8-b37509c250c7 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:20.525626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a38367-cb45-420f-9b1c-72b908012189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:20.526493 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a38367-cb45-420f-9b1c-72b908012189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:20.526777 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a38367-cb45-420f-9b1c-72b908012189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:20.527056 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a38367-cb45-420f-9b1c-72b908012189 admin admin] PATH_INFO: `/v3/projects/535c43e553a144869d08a61dd9c086a4` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:20.528231 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a38367-cb45-420f-9b1c-72b908012189 admin admin] RBAC: Authorizing `identity:delete_project(project_id=535c43e553a144869d08a61dd9c086a4)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:20.530194 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a38367-cb45-420f-9b1c-72b908012189 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:20.565879 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:20.576781 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b3a38367-cb45-420f-9b1c-72b908012189 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '535c43e553a144869d08a61dd9c086a4', 'request_id': 'req-b3a38367-cb45-420f-9b1c-72b908012189'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:20.582701 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b3a38367-cb45-420f-9b1c-72b908012189 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '535c43e553a144869d08a61dd9c086a4', 'request_id': 'req-b3a38367-cb45-420f-9b1c-72b908012189'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:20.589696 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b3a38367-cb45-420f-9b1c-72b908012189 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '535c43e553a144869d08a61dd9c086a4', 'request_id': 'req-b3a38367-cb45-420f-9b1c-72b908012189'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:20.600238 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b3a38367-cb45-420f-9b1c-72b908012189 admin admin] The token cache is being invalidate because project 535c43e553a144869d08a61dd9c086a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:11:20.600526 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b3a38367-cb45-420f-9b1c-72b908012189 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:20.601150 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b3a38367-cb45-420f-9b1c-72b908012189 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:20.602698 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1278/2554] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:11:20 2022] DELETE /identity/v3/projects/535c43e553a144869d08a61dd9c086a4 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:20.607521 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c56ac-c168-4196-8ab8-1cd226862fff None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:20.607811 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c56ac-c168-4196-8ab8-1cd226862fff None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:20.608090 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c56ac-c168-4196-8ab8-1cd226862fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:20.626064 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5a7c56ac-c168-4196-8ab8-1cd226862fff None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:20.627244 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5a7c56ac-c168-4196-8ab8-1cd226862fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:20.685250 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1277/2555] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:20.693071 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3386fb-9bca-4f9e-b292-d4aa8e6179bd None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:20.693366 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3386fb-9bca-4f9e-b292-d4aa8e6179bd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:20.693676 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3386fb-9bca-4f9e-b292-d4aa8e6179bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:20.711476 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7c3386fb-9bca-4f9e-b292-d4aa8e6179bd None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:20.712692 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7c3386fb-9bca-4f9e-b292-d4aa8e6179bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:20.720559 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1279/2556] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:20.724854 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c56ac-c168-4196-8ab8-1cd226862fff None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:20.734995 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c56ac-c168-4196-8ab8-1cd226862fff None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:20.739554 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d5377-4a4a-403c-b175-ef58c880fe67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:20.740193 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d5377-4a4a-403c-b175-ef58c880fe67 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:20.740478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d5377-4a4a-403c-b175-ef58c880fe67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:20.740754 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d5377-4a4a-403c-b175-ef58c880fe67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:20.741471 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5d5377-4a4a-403c-b175-ef58c880fe67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:11:20.741857 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5d5377-4a4a-403c-b175-ef58c880fe67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:20.744066 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5d5377-4a4a-403c-b175-ef58c880fe67 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:20.755364 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1278/2557] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:11:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:11:20.759598 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3386fb-9bca-4f9e-b292-d4aa8e6179bd None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:20.768708 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3386fb-9bca-4f9e-b292-d4aa8e6179bd None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:20.773242 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dbba17-f989-4d66-bcdf-a4439f4daf55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:20.773916 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dbba17-f989-4d66-bcdf-a4439f4daf55 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:20.774208 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dbba17-f989-4d66-bcdf-a4439f4daf55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:20.774485 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dbba17-f989-4d66-bcdf-a4439f4daf55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:20.775185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dbba17-f989-4d66-bcdf-a4439f4daf55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:20.777074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dbba17-f989-4d66-bcdf-a4439f4daf55 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:20.804275 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1280/2558] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:11:20 2022] POST /identity/v3/projects => generated 405 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:11:20.808467 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d5377-4a4a-403c-b175-ef58c880fe67 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:20.818467 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d5377-4a4a-403c-b175-ef58c880fe67 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:20.859115 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:20.875676 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc8443-e80e-431d-8f2f-4d7197394859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:20.876375 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc8443-e80e-431d-8f2f-4d7197394859 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:20.876658 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc8443-e80e-431d-8f2f-4d7197394859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:20.876934 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc8443-e80e-431d-8f2f-4d7197394859 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:20.877658 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcc8443-e80e-431d-8f2f-4d7197394859 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:20.879601 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcc8443-e80e-431d-8f2f-4d7197394859 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:20.898645 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1279/2559] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:11:20 2022] POST /identity/v3/users => generated 303 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:11:20.902804 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dbba17-f989-4d66-bcdf-a4439f4daf55 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:20.912878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dbba17-f989-4d66-bcdf-a4439f4daf55 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:20.918013 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525063a-c00b-4de8-ae71-e4333ea3d940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:20.918670 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525063a-c00b-4de8-ae71-e4333ea3d940 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:20.918958 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525063a-c00b-4de8-ae71-e4333ea3d940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:20.919241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525063a-c00b-4de8-ae71-e4333ea3d940 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:20.919968 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525063a-c00b-4de8-ae71-e4333ea3d940 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:20.921560 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525063a-c00b-4de8-ae71-e4333ea3d940 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:20.931706 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1281/2560] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:11:20.935637 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc8443-e80e-431d-8f2f-4d7197394859 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:20.946324 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc8443-e80e-431d-8f2f-4d7197394859 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:20.951270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29897f-0127-4642-b615-844b698c7b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:20.951930 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29897f-0127-4642-b615-844b698c7b3a admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:20.952213 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29897f-0127-4642-b615-844b698c7b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:20.952488 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29897f-0127-4642-b615-844b698c7b3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:20.953244 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29897f-0127-4642-b615-844b698c7b3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:20.954875 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29897f-0127-4642-b615-844b698c7b3a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:20.965446 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1280/2561] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:11:20.969659 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525063a-c00b-4de8-ae71-e4333ea3d940 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:20.980489 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525063a-c00b-4de8-ae71-e4333ea3d940 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:20.985081 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e2bc66-cae9-4072-875c-71a6cd515ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:20.985994 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e2bc66-cae9-4072-875c-71a6cd515ed6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:20.986374 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e2bc66-cae9-4072-875c-71a6cd515ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:20.986668 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e2bc66-cae9-4072-875c-71a6cd515ed6 admin admin] PATH_INFO: `/v3/projects/2275304b155c40afbfff6861c1956964/users/22e61dfbf6664463898213587917c79c/roles/54195e660fc349978b39d5794765a095` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:21.000887 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e2bc66-cae9-4072-875c-71a6cd515ed6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2275304b155c40afbfff6861c1956964, user_id=22e61dfbf6664463898213587917c79c, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:21.003563 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e2bc66-cae9-4072-875c-71a6cd515ed6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:21.016606 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1282/2562] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:11:20 2022] PUT /identity/v3/projects/2275304b155c40afbfff6861c1956964/users/22e61dfbf6664463898213587917c79c/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:21.027122 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2675d-bbfd-4a81-899b-a0ad1c6a63f3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:21.027409 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2675d-bbfd-4a81-899b-a0ad1c6a63f3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:21.027687 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2675d-bbfd-4a81-899b-a0ad1c6a63f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:21.053900 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-06e2675d-bbfd-4a81-899b-a0ad1c6a63f3 None None] MFA Rules not processed for user `22e61dfbf6664463898213587917c79c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:21.055095 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-06e2675d-bbfd-4a81-899b-a0ad1c6a63f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:21.118349 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1281/2563] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:21 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:21.122639 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e2bc66-cae9-4072-875c-71a6cd515ed6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:21.183606 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e2bc66-cae9-4072-875c-71a6cd515ed6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:21.188579 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c41f08-b1e3-4896-b7cd-514b871e65b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:21.189244 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c41f08-b1e3-4896-b7cd-514b871e65b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:21.189531 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c41f08-b1e3-4896-b7cd-514b871e65b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:21.189844 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c41f08-b1e3-4896-b7cd-514b871e65b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:21.190559 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c41f08-b1e3-4896-b7cd-514b871e65b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:21.192462 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c41f08-b1e3-4896-b7cd-514b871e65b7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:21.218067 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1283/2564] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:11:21 2022] POST /identity/v3/projects => generated 403 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:11:21.222482 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e2675d-bbfd-4a81-899b-a0ad1c6a63f3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:21.234661 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e2675d-bbfd-4a81-899b-a0ad1c6a63f3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:21.295429 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ede2a-35b1-4254-9e56-c028b296291f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:21.296102 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ede2a-35b1-4254-9e56-c028b296291f admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:21.296388 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ede2a-35b1-4254-9e56-c028b296291f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:21.296665 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ede2a-35b1-4254-9e56-c028b296291f admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:21.297366 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173ede2a-35b1-4254-9e56-c028b296291f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:21.299327 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173ede2a-35b1-4254-9e56-c028b296291f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:21.319730 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1282/2565] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:11:21 2022] POST /identity/v3/users => generated 308 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:11:21.323731 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c41f08-b1e3-4896-b7cd-514b871e65b7 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:21.333913 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c41f08-b1e3-4896-b7cd-514b871e65b7 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:21.339009 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ac246-007f-4ea3-a730-e07085050dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:21.339657 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ac246-007f-4ea3-a730-e07085050dca admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:21.339942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ac246-007f-4ea3-a730-e07085050dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:21.340220 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ac246-007f-4ea3-a730-e07085050dca admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:21.340945 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236ac246-007f-4ea3-a730-e07085050dca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:21.342686 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236ac246-007f-4ea3-a730-e07085050dca admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:21.353272 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1284/2566] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:11:21.357230 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ede2a-35b1-4254-9e56-c028b296291f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:21.367365 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ede2a-35b1-4254-9e56-c028b296291f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:21.372014 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4203da-940e-458d-b608-00a551fd9984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:21.372812 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4203da-940e-458d-b608-00a551fd9984 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:21.373095 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4203da-940e-458d-b608-00a551fd9984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:21.373372 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4203da-940e-458d-b608-00a551fd9984 admin admin] PATH_INFO: `/v3/projects/fb385577783a4cd89eb9d506ac00adcf/users/8d2acf0f1e1043388b415a19a0aaa875/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:21.389558 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4203da-940e-458d-b608-00a551fd9984 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb385577783a4cd89eb9d506ac00adcf, user_id=8d2acf0f1e1043388b415a19a0aaa875, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:21.392221 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4203da-940e-458d-b608-00a551fd9984 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:21.405333 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1283/2567] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:11:21 2022] PUT /identity/v3/projects/fb385577783a4cd89eb9d506ac00adcf/users/8d2acf0f1e1043388b415a19a0aaa875/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:21.409388 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ac246-007f-4ea3-a730-e07085050dca admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:21.419582 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ac246-007f-4ea3-a730-e07085050dca admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:21.449370 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:21.476942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6718e3-fa21-48f1-843d-e97814508e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:21.477626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6718e3-fa21-48f1-843d-e97814508e82 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:21.477912 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6718e3-fa21-48f1-843d-e97814508e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:21.478189 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6718e3-fa21-48f1-843d-e97814508e82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:21.478916 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6718e3-fa21-48f1-843d-e97814508e82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:21.480495 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6718e3-fa21-48f1-843d-e97814508e82 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:21.490180 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1285/2568] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:11:21.494903 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4203da-940e-458d-b608-00a551fd9984 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:21.505318 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4203da-940e-458d-b608-00a551fd9984 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:21.511656 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab23ac-e664-4243-bd25-d20c65953577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:21.512563 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab23ac-e664-4243-bd25-d20c65953577 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:21.513079 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab23ac-e664-4243-bd25-d20c65953577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:21.513405 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab23ac-e664-4243-bd25-d20c65953577 admin admin] PATH_INFO: `/v3/projects/fb385577783a4cd89eb9d506ac00adcf/users/8d2acf0f1e1043388b415a19a0aaa875/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:21.516090 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab23ac-e664-4243-bd25-d20c65953577 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb385577783a4cd89eb9d506ac00adcf, user_id=8d2acf0f1e1043388b415a19a0aaa875, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:21.518786 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab23ac-e664-4243-bd25-d20c65953577 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:21.530762 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1284/2569] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:11:21 2022] PUT /identity/v3/projects/fb385577783a4cd89eb9d506ac00adcf/users/8d2acf0f1e1043388b415a19a0aaa875/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:21.542011 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e29260-865b-4688-8148-245eaa906a6c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:21.542308 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e29260-865b-4688-8148-245eaa906a6c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:21.542601 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e29260-865b-4688-8148-245eaa906a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:21.568243 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b0e29260-865b-4688-8148-245eaa906a6c None None] MFA Rules not processed for user `8d2acf0f1e1043388b415a19a0aaa875`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:21.569459 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b0e29260-865b-4688-8148-245eaa906a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:21.637377 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1286/2570] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:21 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:21.647688 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab23ac-e664-4243-bd25-d20c65953577 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:21.650033 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b8ab23ac-e664-4243-bd25-d20c65953577 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:21.656561 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:21.708534 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab23ac-e664-4243-bd25-d20c65953577 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:21.712186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb2410-ccb7-4acb-b23a-cdfa3419b0e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:21.712825 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb2410-ccb7-4acb-b23a-cdfa3419b0e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:21.713107 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb2410-ccb7-4acb-b23a-cdfa3419b0e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:21.713382 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb2410-ccb7-4acb-b23a-cdfa3419b0e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:21.722497 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcb2410-ccb7-4acb-b23a-cdfa3419b0e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:21.723928 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcb2410-ccb7-4acb-b23a-cdfa3419b0e3 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:21.733363 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1285/2571] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:11:21 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:11:23.132463 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e29260-865b-4688-8148-245eaa906a6c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:23.192910 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e29260-865b-4688-8148-245eaa906a6c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:23.197978 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc67b2f-2c87-4973-84b3-342b67cd64ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:23.198837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc67b2f-2c87-4973-84b3-342b67cd64ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:23.199122 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc67b2f-2c87-4973-84b3-342b67cd64ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:23.199397 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc67b2f-2c87-4973-84b3-342b67cd64ce admin admin] PATH_INFO: `/v3/users/22e61dfbf6664463898213587917c79c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:23.200566 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc67b2f-2c87-4973-84b3-342b67cd64ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=22e61dfbf6664463898213587917c79c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:23.202504 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc67b2f-2c87-4973-84b3-342b67cd64ce admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:23.250639 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2cc67b2f-2c87-4973-84b3-342b67cd64ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22e61dfbf6664463898213587917c79c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:23.251295 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2cc67b2f-2c87-4973-84b3-342b67cd64ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22e61dfbf6664463898213587917c79c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:23.251886 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2cc67b2f-2c87-4973-84b3-342b67cd64ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22e61dfbf6664463898213587917c79c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:23.259659 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2cc67b2f-2c87-4973-84b3-342b67cd64ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22e61dfbf6664463898213587917c79c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:23.261719 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:23.268642 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2cc67b2f-2c87-4973-84b3-342b67cd64ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22e61dfbf6664463898213587917c79c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:23.288769 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1287/2572] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:11:23 2022] DELETE /identity/v3/users/22e61dfbf6664463898213587917c79c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:23.293383 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a0f7d-1f0b-4f3c-b879-16bdab1dda43 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:23.293916 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a0f7d-1f0b-4f3c-b879-16bdab1dda43 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:23.294021 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a0f7d-1f0b-4f3c-b879-16bdab1dda43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:23.311027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-381a0f7d-1f0b-4f3c-b879-16bdab1dda43 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:23.312230 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-381a0f7d-1f0b-4f3c-b879-16bdab1dda43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:23.373468 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1286/2573] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:23.384054 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc67b2f-2c87-4973-84b3-342b67cd64ce admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:23.385608 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2cc67b2f-2c87-4973-84b3-342b67cd64ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:23.444549 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc67b2f-2c87-4973-84b3-342b67cd64ce admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:23.448857 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f2c680-0574-426a-810e-71543133f98b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:23.449510 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2c680-0574-426a-810e-71543133f98b service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:23.449825 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2c680-0574-426a-810e-71543133f98b service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:23.450106 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2c680-0574-426a-810e-71543133f98b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:23.459031 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f2c680-0574-426a-810e-71543133f98b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:23.460501 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f2c680-0574-426a-810e-71543133f98b service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:23.470147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1288/2574] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:11:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:11:23.740856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a0f7d-1f0b-4f3c-b879-16bdab1dda43 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:23.742442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-381a0f7d-1f0b-4f3c-b879-16bdab1dda43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:23.754519 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a0f7d-1f0b-4f3c-b879-16bdab1dda43 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:23.759941 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d8303-9a32-4262-bffb-496ca04bf0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:23.760935 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d8303-9a32-4262-bffb-496ca04bf0c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:23.761229 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d8303-9a32-4262-bffb-496ca04bf0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:23.761510 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d8303-9a32-4262-bffb-496ca04bf0c7 admin admin] PATH_INFO: `/v3/projects/2275304b155c40afbfff6861c1956964` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:23.762914 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d8303-9a32-4262-bffb-496ca04bf0c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2275304b155c40afbfff6861c1956964)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:23.764841 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d8303-9a32-4262-bffb-496ca04bf0c7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:23.813077 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cd9d8303-9a32-4262-bffb-496ca04bf0c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2275304b155c40afbfff6861c1956964', 'request_id': 'req-cd9d8303-9a32-4262-bffb-496ca04bf0c7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:23.820727 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cd9d8303-9a32-4262-bffb-496ca04bf0c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2275304b155c40afbfff6861c1956964', 'request_id': 'req-cd9d8303-9a32-4262-bffb-496ca04bf0c7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:23.826326 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cd9d8303-9a32-4262-bffb-496ca04bf0c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2275304b155c40afbfff6861c1956964', 'request_id': 'req-cd9d8303-9a32-4262-bffb-496ca04bf0c7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:23.837308 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cd9d8303-9a32-4262-bffb-496ca04bf0c7 admin admin] The token cache is being invalidate because project 2275304b155c40afbfff6861c1956964 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:11:23.837615 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cd9d8303-9a32-4262-bffb-496ca04bf0c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:23.838286 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cd9d8303-9a32-4262-bffb-496ca04bf0c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:23.839812 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1287/2575] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:11:23 2022] DELETE /identity/v3/projects/2275304b155c40afbfff6861c1956964 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:23.843763 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f2c680-0574-426a-810e-71543133f98b service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:23.854174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f2c680-0574-426a-810e-71543133f98b service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:23.908154 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c49a5c-353d-4fd9-99dc-d387d0705348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:23.908949 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c49a5c-353d-4fd9-99dc-d387d0705348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:23.909235 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c49a5c-353d-4fd9-99dc-d387d0705348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:23.909518 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c49a5c-353d-4fd9-99dc-d387d0705348 admin admin] PATH_INFO: `/v3/users/8d2acf0f1e1043388b415a19a0aaa875` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:23.910729 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c49a5c-353d-4fd9-99dc-d387d0705348 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d2acf0f1e1043388b415a19a0aaa875)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:23.912648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c49a5c-353d-4fd9-99dc-d387d0705348 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:23.962905 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-25c49a5c-353d-4fd9-99dc-d387d0705348 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d2acf0f1e1043388b415a19a0aaa875'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:23.963566 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-25c49a5c-353d-4fd9-99dc-d387d0705348 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d2acf0f1e1043388b415a19a0aaa875'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:23.964143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-25c49a5c-353d-4fd9-99dc-d387d0705348 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d2acf0f1e1043388b415a19a0aaa875'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:23.971380 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-25c49a5c-353d-4fd9-99dc-d387d0705348 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d2acf0f1e1043388b415a19a0aaa875'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:23.979098 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-25c49a5c-353d-4fd9-99dc-d387d0705348 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d2acf0f1e1043388b415a19a0aaa875'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:23.989889 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:23.998278 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1289/2576] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:11:23 2022] DELETE /identity/v3/users/8d2acf0f1e1043388b415a19a0aaa875 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:24.276252 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d8303-9a32-4262-bffb-496ca04bf0c7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:24.277854 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cd9d8303-9a32-4262-bffb-496ca04bf0c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:24.292367 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d8303-9a32-4262-bffb-496ca04bf0c7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:24.322469 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:24.350432 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d264f4b1-a88d-4fbf-a8c4-45c2be148b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:24.351293 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d264f4b1-a88d-4fbf-a8c4-45c2be148b04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:24.352114 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d264f4b1-a88d-4fbf-a8c4-45c2be148b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:24.352381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d264f4b1-a88d-4fbf-a8c4-45c2be148b04 admin admin] PATH_INFO: `/v3/projects/fb385577783a4cd89eb9d506ac00adcf` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:24.353587 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d264f4b1-a88d-4fbf-a8c4-45c2be148b04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb385577783a4cd89eb9d506ac00adcf)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:24.355510 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d264f4b1-a88d-4fbf-a8c4-45c2be148b04 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:24.403664 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d264f4b1-a88d-4fbf-a8c4-45c2be148b04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb385577783a4cd89eb9d506ac00adcf', 'request_id': 'req-d264f4b1-a88d-4fbf-a8c4-45c2be148b04'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:24.410590 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d264f4b1-a88d-4fbf-a8c4-45c2be148b04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb385577783a4cd89eb9d506ac00adcf', 'request_id': 'req-d264f4b1-a88d-4fbf-a8c4-45c2be148b04'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:24.416841 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d264f4b1-a88d-4fbf-a8c4-45c2be148b04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb385577783a4cd89eb9d506ac00adcf', 'request_id': 'req-d264f4b1-a88d-4fbf-a8c4-45c2be148b04'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:24.429375 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d264f4b1-a88d-4fbf-a8c4-45c2be148b04 admin admin] The token cache is being invalidate because project fb385577783a4cd89eb9d506ac00adcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:11:24.429868 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d264f4b1-a88d-4fbf-a8c4-45c2be148b04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:24.430740 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d264f4b1-a88d-4fbf-a8c4-45c2be148b04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:24.432576 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1288/2577] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:11:24 2022] DELETE /identity/v3/projects/fb385577783a4cd89eb9d506ac00adcf => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:11:24.437399 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b8839-28fe-4f7d-93ac-08d106059748 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:24.437898 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b8839-28fe-4f7d-93ac-08d106059748 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:24.438128 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b8839-28fe-4f7d-93ac-08d106059748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:24.458873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b73b8839-28fe-4f7d-93ac-08d106059748 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:24.460434 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b73b8839-28fe-4f7d-93ac-08d106059748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:24.525924 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1290/2578] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:24.533462 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22134889-814f-479f-bb8b-119b678f2809 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:24.533778 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22134889-814f-479f-bb8b-119b678f2809 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:24.534060 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22134889-814f-479f-bb8b-119b678f2809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:24.552389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-22134889-814f-479f-bb8b-119b678f2809 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:24.553617 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-22134889-814f-479f-bb8b-119b678f2809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:24.562696 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1289/2579] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:24.567106 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b8839-28fe-4f7d-93ac-08d106059748 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:24.577101 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b8839-28fe-4f7d-93ac-08d106059748 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:24.581765 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd595ff-b505-40ac-9b2d-6bbc5309484d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:24.582423 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd595ff-b505-40ac-9b2d-6bbc5309484d admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:24.582711 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd595ff-b505-40ac-9b2d-6bbc5309484d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:24.582993 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd595ff-b505-40ac-9b2d-6bbc5309484d admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:24.583724 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd595ff-b505-40ac-9b2d-6bbc5309484d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:11:24.584036 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd595ff-b505-40ac-9b2d-6bbc5309484d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:24.585660 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd595ff-b505-40ac-9b2d-6bbc5309484d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:24.597396 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1291/2580] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:11:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:11:24.602385 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22134889-814f-479f-bb8b-119b678f2809 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:24.611774 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22134889-814f-479f-bb8b-119b678f2809 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:24.616481 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081146bb-e64c-4268-aa4e-5e708b282039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:24.617123 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081146bb-e64c-4268-aa4e-5e708b282039 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:24.617481 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081146bb-e64c-4268-aa4e-5e708b282039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:24.617789 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081146bb-e64c-4268-aa4e-5e708b282039 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:24.618473 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081146bb-e64c-4268-aa4e-5e708b282039 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:24.620395 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081146bb-e64c-4268-aa4e-5e708b282039 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:24.645558 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1290/2581] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:11:24 2022] POST /identity/v3/projects => generated 405 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:11:24.650276 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd595ff-b505-40ac-9b2d-6bbc5309484d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:24.660405 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd595ff-b505-40ac-9b2d-6bbc5309484d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:24.718122 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049fca0-86cb-4862-80cd-1e6da00dfe07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:24.718796 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049fca0-86cb-4862-80cd-1e6da00dfe07 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:24.719085 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049fca0-86cb-4862-80cd-1e6da00dfe07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:24.719368 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049fca0-86cb-4862-80cd-1e6da00dfe07 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:24.720088 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e049fca0-86cb-4862-80cd-1e6da00dfe07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:24.722318 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e049fca0-86cb-4862-80cd-1e6da00dfe07 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:24.740490 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1292/2582] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:11:24 2022] POST /identity/v3/users => generated 303 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:11:24.745134 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081146bb-e64c-4268-aa4e-5e708b282039 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:24.755897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081146bb-e64c-4268-aa4e-5e708b282039 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:24.761076 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c095a04a-8bbd-456f-88e3-5bfc393fdcab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:24.761751 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095a04a-8bbd-456f-88e3-5bfc393fdcab admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:24.762044 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095a04a-8bbd-456f-88e3-5bfc393fdcab admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:24.762318 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095a04a-8bbd-456f-88e3-5bfc393fdcab admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:24.763038 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c095a04a-8bbd-456f-88e3-5bfc393fdcab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:24.764670 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c095a04a-8bbd-456f-88e3-5bfc393fdcab admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:24.775771 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1291/2583] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:11:24.779781 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049fca0-86cb-4862-80cd-1e6da00dfe07 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:24.789681 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049fca0-86cb-4862-80cd-1e6da00dfe07 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:24.794251 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb119d-6a86-4973-86a5-d84186cdabdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:24.794909 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb119d-6a86-4973-86a5-d84186cdabdb admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:24.795192 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb119d-6a86-4973-86a5-d84186cdabdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:24.795471 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb119d-6a86-4973-86a5-d84186cdabdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:24.796200 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efb119d-6a86-4973-86a5-d84186cdabdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:24.797849 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efb119d-6a86-4973-86a5-d84186cdabdb admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:24.808036 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1293/2584] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:11:24.812053 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c095a04a-8bbd-456f-88e3-5bfc393fdcab admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:24.822332 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c095a04a-8bbd-456f-88e3-5bfc393fdcab admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:24.827022 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70231df-461e-4007-9345-6b92f6a14bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:24.827888 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70231df-461e-4007-9345-6b92f6a14bbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:24.828171 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70231df-461e-4007-9345-6b92f6a14bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:24.828454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70231df-461e-4007-9345-6b92f6a14bbe admin admin] PATH_INFO: `/v3/projects/e72d5ef851604f519ba2e11714542ebf/users/d5d043f7690042d9b2a59510c1c45293/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:24.843461 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70231df-461e-4007-9345-6b92f6a14bbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=e72d5ef851604f519ba2e11714542ebf, user_id=d5d043f7690042d9b2a59510c1c45293, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:24.846153 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70231df-461e-4007-9345-6b92f6a14bbe admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:24.860869 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1292/2585] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:11:24 2022] PUT /identity/v3/projects/e72d5ef851604f519ba2e11714542ebf/users/d5d043f7690042d9b2a59510c1c45293/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:24.866178 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e389a15-3456-4024-aed6-267196e9b8bc None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:24.866475 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e389a15-3456-4024-aed6-267196e9b8bc None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:24.866754 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e389a15-3456-4024-aed6-267196e9b8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:24.884455 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0e389a15-3456-4024-aed6-267196e9b8bc None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:24.885741 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0e389a15-3456-4024-aed6-267196e9b8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:24.953421 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1294/2586] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:24.963678 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70231df-461e-4007-9345-6b92f6a14bbe admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:24.965365 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d70231df-461e-4007-9345-6b92f6a14bbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:25.021417 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70231df-461e-4007-9345-6b92f6a14bbe admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:25.025500 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a6458-89c2-4e3c-b559-3ab27a17e982 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:25.026167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a6458-89c2-4e3c-b559-3ab27a17e982 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:25.026611 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a6458-89c2-4e3c-b559-3ab27a17e982 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:25.026954 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a6458-89c2-4e3c-b559-3ab27a17e982 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:25.036708 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a6458-89c2-4e3c-b559-3ab27a17e982 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:25.038228 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a6458-89c2-4e3c-b559-3ab27a17e982 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:25.043838 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:25.049278 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1293/2587] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:11:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:11:25.658137 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7c66f-1f83-4efc-810d-fb8ad57441d1 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:25.658944 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7c66f-1f83-4efc-810d-fb8ad57441d1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:25.659436 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7c66f-1f83-4efc-810d-fb8ad57441d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:25.667526 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:25.693979 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f4a7c66f-1f83-4efc-810d-fb8ad57441d1 None None] MFA Rules not processed for user `d5d043f7690042d9b2a59510c1c45293`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:25.695338 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f4a7c66f-1f83-4efc-810d-fb8ad57441d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:25.769286 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1295/2588] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:25 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:25.774215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a6458-89c2-4e3c-b559-3ab27a17e982 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:25.784906 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a6458-89c2-4e3c-b559-3ab27a17e982 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:25.790191 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1d597-10dd-4ef2-80b7-89238706e98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:25.791019 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1d597-10dd-4ef2-80b7-89238706e98c admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:25.791413 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1d597-10dd-4ef2-80b7-89238706e98c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:25.791783 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1d597-10dd-4ef2-80b7-89238706e98c admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:25.792573 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d1d597-10dd-4ef2-80b7-89238706e98c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:25.794641 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d1d597-10dd-4ef2-80b7-89238706e98c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:25.827703 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1294/2589] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:11:25 2022] POST /identity/v3/projects => generated 405 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:11:25.831946 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7c66f-1f83-4efc-810d-fb8ad57441d1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:25.844780 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7c66f-1f83-4efc-810d-fb8ad57441d1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:25.917244 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a4fdc-eb80-41e0-9ca9-fde0a82d84a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:25.918142 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a4fdc-eb80-41e0-9ca9-fde0a82d84a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:25.918505 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a4fdc-eb80-41e0-9ca9-fde0a82d84a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:25.918858 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a4fdc-eb80-41e0-9ca9-fde0a82d84a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:25.919653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a4fdc-eb80-41e0-9ca9-fde0a82d84a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:25.921720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a4fdc-eb80-41e0-9ca9-fde0a82d84a3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:25.945970 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1296/2590] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:11:25 2022] POST /identity/v3/users => generated 309 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:11:25.951787 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1d597-10dd-4ef2-80b7-89238706e98c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:25.963805 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1d597-10dd-4ef2-80b7-89238706e98c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:25.969219 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799d672-381d-4fd7-85ed-ecc304f238d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:25.970055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799d672-381d-4fd7-85ed-ecc304f238d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:25.970518 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799d672-381d-4fd7-85ed-ecc304f238d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:25.970993 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799d672-381d-4fd7-85ed-ecc304f238d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:25.972520 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5799d672-381d-4fd7-85ed-ecc304f238d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:25.975202 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5799d672-381d-4fd7-85ed-ecc304f238d7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:25.991957 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1295/2591] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:11:25.996341 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a4fdc-eb80-41e0-9ca9-fde0a82d84a3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:26.007216 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a4fdc-eb80-41e0-9ca9-fde0a82d84a3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:26.012601 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d795fe6-3716-4666-ace4-cdc809d6a121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:26.013438 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d795fe6-3716-4666-ace4-cdc809d6a121 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:26.013830 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d795fe6-3716-4666-ace4-cdc809d6a121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:26.014117 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d795fe6-3716-4666-ace4-cdc809d6a121 admin admin] PATH_INFO: `/v3/projects/9b6a53503bdc49bb87c89521b24a2b86/users/d23cf296650b42168433c291925287a5/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:26.029470 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d795fe6-3716-4666-ace4-cdc809d6a121 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b6a53503bdc49bb87c89521b24a2b86, user_id=d23cf296650b42168433c291925287a5, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:26.032214 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d795fe6-3716-4666-ace4-cdc809d6a121 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:26.045902 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1297/2592] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:11:25 2022] PUT /identity/v3/projects/9b6a53503bdc49bb87c89521b24a2b86/users/d23cf296650b42168433c291925287a5/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:26.049993 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799d672-381d-4fd7-85ed-ecc304f238d7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:26.060002 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799d672-381d-4fd7-85ed-ecc304f238d7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:26.116617 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1233c3eb-b3d7-49fa-999e-34a91443ff3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:26.117338 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233c3eb-b3d7-49fa-999e-34a91443ff3a admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:26.117651 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233c3eb-b3d7-49fa-999e-34a91443ff3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:26.117931 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233c3eb-b3d7-49fa-999e-34a91443ff3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:26.118661 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1233c3eb-b3d7-49fa-999e-34a91443ff3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:26.120458 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1233c3eb-b3d7-49fa-999e-34a91443ff3a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:26.130210 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1296/2593] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:11:26 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:11:26.135013 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d795fe6-3716-4666-ace4-cdc809d6a121 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:26.145470 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d795fe6-3716-4666-ace4-cdc809d6a121 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:26.150846 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1c3d50-53ac-425a-b6ed-93da5eb468fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:26.151720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c3d50-53ac-425a-b6ed-93da5eb468fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:26.152006 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c3d50-53ac-425a-b6ed-93da5eb468fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:26.152309 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c3d50-53ac-425a-b6ed-93da5eb468fe admin admin] PATH_INFO: `/v3/projects/9b6a53503bdc49bb87c89521b24a2b86/users/d23cf296650b42168433c291925287a5/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:26.154402 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1c3d50-53ac-425a-b6ed-93da5eb468fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b6a53503bdc49bb87c89521b24a2b86, user_id=d23cf296650b42168433c291925287a5, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:26.157045 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1c3d50-53ac-425a-b6ed-93da5eb468fe admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:26.167076 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1298/2594] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:11:26 2022] PUT /identity/v3/projects/9b6a53503bdc49bb87c89521b24a2b86/users/d23cf296650b42168433c291925287a5/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:26.805190 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8cc88-d136-44d5-a5d4-e5199a8eb3a2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:26.805786 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8cc88-d136-44d5-a5d4-e5199a8eb3a2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:26.805786 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8cc88-d136-44d5-a5d4-e5199a8eb3a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:26.832205 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c0f8cc88-d136-44d5-a5d4-e5199a8eb3a2 None None] MFA Rules not processed for user `d23cf296650b42168433c291925287a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:26.833444 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c0f8cc88-d136-44d5-a5d4-e5199a8eb3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:26.907830 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1297/2595] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:26 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:27.517815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb4b43-a446-42f1-b9ee-b66d2f11d3d0 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:27.518267 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb4b43-a446-42f1-b9ee-b66d2f11d3d0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:27.518399 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb4b43-a446-42f1-b9ee-b66d2f11d3d0 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:27.519770 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1299/2596] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jul 21 16:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 16:11:27.530606 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9bb0c8-68fa-422a-acac-95433f02bf64 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:27.530973 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9bb0c8-68fa-422a-acac-95433f02bf64 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:27.531330 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9bb0c8-68fa-422a-acac-95433f02bf64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:27.549814 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8d9bb0c8-68fa-422a-acac-95433f02bf64 None None] MFA Rules not processed for user `822d9d3521db4a9688f51c6313c7dd74`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:27.551082 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8d9bb0c8-68fa-422a-acac-95433f02bf64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:27.601894 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:27.623707 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1298/2597] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jul 21 16:11:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:27.634623 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb4b43-a446-42f1-b9ee-b66d2f11d3d0 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:27.636320 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0dbb4b43-a446-42f1-b9ee-b66d2f11d3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:27.653096 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb4b43-a446-42f1-b9ee-b66d2f11d3d0 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:27.658143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d52c7-e8b8-4d8a-8bd0-0c9eeca97a10 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822d9d3521db4a9688f51c6313c7dd74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:27.658786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d52c7-e8b8-4d8a-8bd0-0c9eeca97a10 service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:27.659074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d52c7-e8b8-4d8a-8bd0-0c9eeca97a10 service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:27.659368 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d52c7-e8b8-4d8a-8bd0-0c9eeca97a10 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:27.668547 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d52c7-e8b8-4d8a-8bd0-0c9eeca97a10 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:27.670368 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d52c7-e8b8-4d8a-8bd0-0c9eeca97a10 service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:27.681866 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1300/2598] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:11:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:11:28.561452 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9bb0c8-68fa-422a-acac-95433f02bf64 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:28.623565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9bb0c8-68fa-422a-acac-95433f02bf64 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:28.627966 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174ef39-b3fb-46aa-a607-43784ff20b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:28.628833 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174ef39-b3fb-46aa-a607-43784ff20b99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:28.629114 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174ef39-b3fb-46aa-a607-43784ff20b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:28.629388 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174ef39-b3fb-46aa-a607-43784ff20b99 admin admin] PATH_INFO: `/v3/users/d5d043f7690042d9b2a59510c1c45293` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:28.630601 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2174ef39-b3fb-46aa-a607-43784ff20b99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5d043f7690042d9b2a59510c1c45293)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:28.632545 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2174ef39-b3fb-46aa-a607-43784ff20b99 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:28.692983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2174ef39-b3fb-46aa-a607-43784ff20b99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5d043f7690042d9b2a59510c1c45293'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:28.693664 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2174ef39-b3fb-46aa-a607-43784ff20b99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5d043f7690042d9b2a59510c1c45293'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:28.694268 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2174ef39-b3fb-46aa-a607-43784ff20b99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5d043f7690042d9b2a59510c1c45293'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:28.701821 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2174ef39-b3fb-46aa-a607-43784ff20b99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5d043f7690042d9b2a59510c1c45293'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:28.709166 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2174ef39-b3fb-46aa-a607-43784ff20b99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5d043f7690042d9b2a59510c1c45293'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:28.726504 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1299/2599] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:11:28 2022] DELETE /identity/v3/users/d5d043f7690042d9b2a59510c1c45293 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:28.899123 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d52c7-e8b8-4d8a-8bd0-0c9eeca97a10 service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:28.915361 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d52c7-e8b8-4d8a-8bd0-0c9eeca97a10 service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:28.947308 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:28.986939 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e731e80-cb16-4e93-8b60-61655a80fc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:28.987825 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e731e80-cb16-4e93-8b60-61655a80fc4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:28.988109 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e731e80-cb16-4e93-8b60-61655a80fc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:28.988396 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e731e80-cb16-4e93-8b60-61655a80fc4f admin admin] PATH_INFO: `/v3/projects/e72d5ef851604f519ba2e11714542ebf` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:28.989732 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e731e80-cb16-4e93-8b60-61655a80fc4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e72d5ef851604f519ba2e11714542ebf)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:28.991656 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e731e80-cb16-4e93-8b60-61655a80fc4f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:29.038237 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2e731e80-cb16-4e93-8b60-61655a80fc4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e72d5ef851604f519ba2e11714542ebf', 'request_id': 'req-2e731e80-cb16-4e93-8b60-61655a80fc4f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:29.045197 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2e731e80-cb16-4e93-8b60-61655a80fc4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e72d5ef851604f519ba2e11714542ebf', 'request_id': 'req-2e731e80-cb16-4e93-8b60-61655a80fc4f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:29.052431 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2e731e80-cb16-4e93-8b60-61655a80fc4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e72d5ef851604f519ba2e11714542ebf', 'request_id': 'req-2e731e80-cb16-4e93-8b60-61655a80fc4f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:29.063626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2e731e80-cb16-4e93-8b60-61655a80fc4f admin admin] The token cache is being invalidate because project e72d5ef851604f519ba2e11714542ebf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:11:29.063921 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2e731e80-cb16-4e93-8b60-61655a80fc4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:29.064558 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2e731e80-cb16-4e93-8b60-61655a80fc4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:29.066149 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1301/2600] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:11:28 2022] DELETE /identity/v3/projects/e72d5ef851604f519ba2e11714542ebf => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:29.070454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174ef39-b3fb-46aa-a607-43784ff20b99 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:29.071845 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2174ef39-b3fb-46aa-a607-43784ff20b99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:29.084963 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174ef39-b3fb-46aa-a607-43784ff20b99 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:29.140831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae37e3-f394-49ab-b4e0-b556ccc94b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:29.141785 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae37e3-f394-49ab-b4e0-b556ccc94b35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:29.142020 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae37e3-f394-49ab-b4e0-b556ccc94b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:29.142300 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae37e3-f394-49ab-b4e0-b556ccc94b35 admin admin] PATH_INFO: `/v3/users/d23cf296650b42168433c291925287a5` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:29.143499 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae37e3-f394-49ab-b4e0-b556ccc94b35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d23cf296650b42168433c291925287a5)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:29.145440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae37e3-f394-49ab-b4e0-b556ccc94b35 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:29.150598 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:29.193863 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-deae37e3-f394-49ab-b4e0-b556ccc94b35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd23cf296650b42168433c291925287a5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:29.194511 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-deae37e3-f394-49ab-b4e0-b556ccc94b35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd23cf296650b42168433c291925287a5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:29.195691 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-deae37e3-f394-49ab-b4e0-b556ccc94b35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd23cf296650b42168433c291925287a5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:29.202826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-deae37e3-f394-49ab-b4e0-b556ccc94b35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd23cf296650b42168433c291925287a5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:29.210010 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-deae37e3-f394-49ab-b4e0-b556ccc94b35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd23cf296650b42168433c291925287a5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:29.228442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1300/2601] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:11:29 2022] DELETE /identity/v3/users/d23cf296650b42168433c291925287a5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:29.401110 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e731e80-cb16-4e93-8b60-61655a80fc4f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:29.403006 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2e731e80-cb16-4e93-8b60-61655a80fc4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:29.416271 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e731e80-cb16-4e93-8b60-61655a80fc4f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:29.474291 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f6b08-113c-40a9-95f4-8fb9967b2db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:29.475155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f6b08-113c-40a9-95f4-8fb9967b2db2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:29.475468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f6b08-113c-40a9-95f4-8fb9967b2db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:29.475754 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f6b08-113c-40a9-95f4-8fb9967b2db2 admin admin] PATH_INFO: `/v3/projects/9b6a53503bdc49bb87c89521b24a2b86` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:29.476966 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5f6b08-113c-40a9-95f4-8fb9967b2db2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b6a53503bdc49bb87c89521b24a2b86)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:29.478933 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5f6b08-113c-40a9-95f4-8fb9967b2db2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:29.524099 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4e5f6b08-113c-40a9-95f4-8fb9967b2db2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b6a53503bdc49bb87c89521b24a2b86', 'request_id': 'req-4e5f6b08-113c-40a9-95f4-8fb9967b2db2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:29.530051 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4e5f6b08-113c-40a9-95f4-8fb9967b2db2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b6a53503bdc49bb87c89521b24a2b86', 'request_id': 'req-4e5f6b08-113c-40a9-95f4-8fb9967b2db2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:29.537762 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4e5f6b08-113c-40a9-95f4-8fb9967b2db2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b6a53503bdc49bb87c89521b24a2b86', 'request_id': 'req-4e5f6b08-113c-40a9-95f4-8fb9967b2db2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:29.539757 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:29.550613 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4e5f6b08-113c-40a9-95f4-8fb9967b2db2 admin admin] The token cache is being invalidate because project 9b6a53503bdc49bb87c89521b24a2b86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:11:29.550900 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4e5f6b08-113c-40a9-95f4-8fb9967b2db2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:29.551528 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4e5f6b08-113c-40a9-95f4-8fb9967b2db2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:29.553042 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1302/2602] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:11:29 2022] DELETE /identity/v3/projects/9b6a53503bdc49bb87c89521b24a2b86 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:29.558492 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffb8df-d9d4-4154-8aa1-122a9792a71e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:29.558819 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffb8df-d9d4-4154-8aa1-122a9792a71e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:29.559101 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffb8df-d9d4-4154-8aa1-122a9792a71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:29.580442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4dffb8df-d9d4-4154-8aa1-122a9792a71e None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:29.581857 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4dffb8df-d9d4-4154-8aa1-122a9792a71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:29.641957 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1301/2603] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:29.650445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409a181-72b7-48de-83a7-ef412c951cb0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:29.650679 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409a181-72b7-48de-83a7-ef412c951cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:29.650959 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409a181-72b7-48de-83a7-ef412c951cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:29.668729 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2409a181-72b7-48de-83a7-ef412c951cb0 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:29.669963 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2409a181-72b7-48de-83a7-ef412c951cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:29.678348 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1303/2604] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:29.682453 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffb8df-d9d4-4154-8aa1-122a9792a71e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:29.692063 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffb8df-d9d4-4154-8aa1-122a9792a71e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:29.697198 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e09822-1b34-469d-99d0-7888ddaf2b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:29.697885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e09822-1b34-469d-99d0-7888ddaf2b5b admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:29.698166 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e09822-1b34-469d-99d0-7888ddaf2b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:29.698440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e09822-1b34-469d-99d0-7888ddaf2b5b admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:29.699271 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e09822-1b34-469d-99d0-7888ddaf2b5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:11:29.699599 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e09822-1b34-469d-99d0-7888ddaf2b5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:29.701211 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e09822-1b34-469d-99d0-7888ddaf2b5b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:29.713475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1302/2605] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:11:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:11:29.717784 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2409a181-72b7-48de-83a7-ef412c951cb0 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:29.727757 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2409a181-72b7-48de-83a7-ef412c951cb0 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:29.737108 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e4592-3064-4fa6-ae19-effc12f9173f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:29.737802 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e4592-3064-4fa6-ae19-effc12f9173f admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:29.738087 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e4592-3064-4fa6-ae19-effc12f9173f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:29.738507 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e4592-3064-4fa6-ae19-effc12f9173f admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:29.739320 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707e4592-3064-4fa6-ae19-effc12f9173f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:29.741230 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707e4592-3064-4fa6-ae19-effc12f9173f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:29.767329 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1304/2606] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:11:29 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:11:29.771413 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e09822-1b34-469d-99d0-7888ddaf2b5b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:29.782107 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e09822-1b34-469d-99d0-7888ddaf2b5b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:29.837225 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8687b02d-8b36-4c13-8431-eca395ff809c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:29.837916 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8687b02d-8b36-4c13-8431-eca395ff809c admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:29.838205 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8687b02d-8b36-4c13-8431-eca395ff809c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:29.838485 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8687b02d-8b36-4c13-8431-eca395ff809c admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:29.839171 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8687b02d-8b36-4c13-8431-eca395ff809c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:29.841133 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8687b02d-8b36-4c13-8431-eca395ff809c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:29.856295 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:29.861984 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1303/2607] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:11:29 2022] POST /identity/v3/users => generated 282 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:11:29.865872 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e4592-3064-4fa6-ae19-effc12f9173f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:29.876450 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e4592-3064-4fa6-ae19-effc12f9173f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:29.881035 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90662ea-d59e-43e5-913b-1e5ec0bf1e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:29.881732 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90662ea-d59e-43e5-913b-1e5ec0bf1e07 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:29.882020 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90662ea-d59e-43e5-913b-1e5ec0bf1e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:29.882319 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90662ea-d59e-43e5-913b-1e5ec0bf1e07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:29.883538 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90662ea-d59e-43e5-913b-1e5ec0bf1e07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:29.885133 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90662ea-d59e-43e5-913b-1e5ec0bf1e07 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:29.895558 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1305/2608] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:11:29.900147 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8687b02d-8b36-4c13-8431-eca395ff809c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:29.910122 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8687b02d-8b36-4c13-8431-eca395ff809c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:29.914656 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7234586f-de47-4817-900d-f654cd96dad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:29.915310 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234586f-de47-4817-900d-f654cd96dad7 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:29.915594 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234586f-de47-4817-900d-f654cd96dad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:29.915871 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234586f-de47-4817-900d-f654cd96dad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:29.916591 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7234586f-de47-4817-900d-f654cd96dad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:29.918230 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7234586f-de47-4817-900d-f654cd96dad7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:29.930210 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1304/2609] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:11:29.934248 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90662ea-d59e-43e5-913b-1e5ec0bf1e07 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:29.944290 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90662ea-d59e-43e5-913b-1e5ec0bf1e07 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:29.950126 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f69801-db34-4117-9319-0a39b5dbd271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:29.951095 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f69801-db34-4117-9319-0a39b5dbd271 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:29.951404 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f69801-db34-4117-9319-0a39b5dbd271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:29.951757 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f69801-db34-4117-9319-0a39b5dbd271 admin admin] PATH_INFO: `/v3/projects/44c8621fee86480a9c21316b3ad98dc3/users/a98a995ea9184c4389b3160d1cc4a95b/roles/54195e660fc349978b39d5794765a095` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:29.966965 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f69801-db34-4117-9319-0a39b5dbd271 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44c8621fee86480a9c21316b3ad98dc3, user_id=a98a995ea9184c4389b3160d1cc4a95b, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:29.969652 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f69801-db34-4117-9319-0a39b5dbd271 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:29.983200 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1306/2610] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:11:29 2022] PUT /identity/v3/projects/44c8621fee86480a9c21316b3ad98dc3/users/a98a995ea9184c4389b3160d1cc4a95b/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:29.994988 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98518e-7505-4c5b-b187-583b187addeb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:29.995278 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98518e-7505-4c5b-b187-583b187addeb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:29.995580 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98518e-7505-4c5b-b187-583b187addeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:30.024664 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5a98518e-7505-4c5b-b187-583b187addeb None None] MFA Rules not processed for user `a98a995ea9184c4389b3160d1cc4a95b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:30.025911 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5a98518e-7505-4c5b-b187-583b187addeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:30.088679 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1305/2611] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:29 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:30.093276 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f69801-db34-4117-9319-0a39b5dbd271 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:30.157417 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f69801-db34-4117-9319-0a39b5dbd271 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:30.162170 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28572f9-3949-43f0-a5e1-7b4d85bfc9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:30.162770 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28572f9-3949-43f0-a5e1-7b4d85bfc9a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:30.163061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28572f9-3949-43f0-a5e1-7b4d85bfc9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:30.163345 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28572f9-3949-43f0-a5e1-7b4d85bfc9a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:30.164051 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28572f9-3949-43f0-a5e1-7b4d85bfc9a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:30.166013 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28572f9-3949-43f0-a5e1-7b4d85bfc9a1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:30.191054 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1307/2612] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:11:30 2022] POST /identity/v3/projects => generated 365 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:11:30.195836 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98518e-7505-4c5b-b187-583b187addeb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:30.206594 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98518e-7505-4c5b-b187-583b187addeb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:30.267315 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e8fa80-9950-4786-abd1-116e51bc6066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:30.267989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e8fa80-9950-4786-abd1-116e51bc6066 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:30.268272 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e8fa80-9950-4786-abd1-116e51bc6066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:30.268547 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e8fa80-9950-4786-abd1-116e51bc6066 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:30.269245 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e8fa80-9950-4786-abd1-116e51bc6066 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:30.271478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e8fa80-9950-4786-abd1-116e51bc6066 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:30.292471 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1306/2613] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:11:30 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:11:30.296524 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28572f9-3949-43f0-a5e1-7b4d85bfc9a1 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:30.307806 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28572f9-3949-43f0-a5e1-7b4d85bfc9a1 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:30.312393 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f161317-faa0-4cca-a68f-0e76672d92c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:30.313059 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f161317-faa0-4cca-a68f-0e76672d92c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:30.313367 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f161317-faa0-4cca-a68f-0e76672d92c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:30.313677 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f161317-faa0-4cca-a68f-0e76672d92c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:30.314412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f161317-faa0-4cca-a68f-0e76672d92c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:30.316045 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f161317-faa0-4cca-a68f-0e76672d92c7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:30.329830 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1308/2614] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:11:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:11:30.334075 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e8fa80-9950-4786-abd1-116e51bc6066 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:30.344176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e8fa80-9950-4786-abd1-116e51bc6066 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:30.349153 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e3919-2963-4134-8bf8-eb028c7361a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:30.350112 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e3919-2963-4134-8bf8-eb028c7361a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:30.350412 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e3919-2963-4134-8bf8-eb028c7361a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:30.350704 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e3919-2963-4134-8bf8-eb028c7361a1 admin admin] PATH_INFO: `/v3/projects/d0638371ee0441ccb0b5803a4ac41e27/users/8936e4509a044faabfa7e9443b4e10db/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:30.366072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e3919-2963-4134-8bf8-eb028c7361a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0638371ee0441ccb0b5803a4ac41e27, user_id=8936e4509a044faabfa7e9443b4e10db, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:30.369514 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e3919-2963-4134-8bf8-eb028c7361a1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:30.382927 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1307/2615] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:11:30 2022] PUT /identity/v3/projects/d0638371ee0441ccb0b5803a4ac41e27/users/8936e4509a044faabfa7e9443b4e10db/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:30.386873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f161317-faa0-4cca-a68f-0e76672d92c7 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:30.397919 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f161317-faa0-4cca-a68f-0e76672d92c7 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:30.456949 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0288d19-425a-4640-8ee6-1a6564ec156f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:30.457777 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0288d19-425a-4640-8ee6-1a6564ec156f admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:30.457969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0288d19-425a-4640-8ee6-1a6564ec156f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:30.458264 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0288d19-425a-4640-8ee6-1a6564ec156f admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:30.458999 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0288d19-425a-4640-8ee6-1a6564ec156f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:30.461243 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0288d19-425a-4640-8ee6-1a6564ec156f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:30.463765 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:30.472395 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1309/2616] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:11:30 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:11:30.476753 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e3919-2963-4134-8bf8-eb028c7361a1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:30.488014 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e3919-2963-4134-8bf8-eb028c7361a1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:30.493264 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee38cb-e781-4d3e-b248-a86ecdc41743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:30.494265 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee38cb-e781-4d3e-b248-a86ecdc41743 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:30.494592 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee38cb-e781-4d3e-b248-a86ecdc41743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:30.494930 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee38cb-e781-4d3e-b248-a86ecdc41743 admin admin] PATH_INFO: `/v3/projects/d0638371ee0441ccb0b5803a4ac41e27/users/8936e4509a044faabfa7e9443b4e10db/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:30.497164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfee38cb-e781-4d3e-b248-a86ecdc41743 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0638371ee0441ccb0b5803a4ac41e27, user_id=8936e4509a044faabfa7e9443b4e10db, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:30.499922 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfee38cb-e781-4d3e-b248-a86ecdc41743 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:30.511859 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1308/2617] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:11:30 2022] PUT /identity/v3/projects/d0638371ee0441ccb0b5803a4ac41e27/users/8936e4509a044faabfa7e9443b4e10db/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:30.522105 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f45d5-2c63-4f5e-aae0-de7deae29680 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:30.522406 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f45d5-2c63-4f5e-aae0-de7deae29680 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:30.522691 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f45d5-2c63-4f5e-aae0-de7deae29680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:30.554642 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f34f45d5-2c63-4f5e-aae0-de7deae29680 None None] MFA Rules not processed for user `8936e4509a044faabfa7e9443b4e10db`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:30.556082 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f34f45d5-2c63-4f5e-aae0-de7deae29680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:30.644438 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1310/2618] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:30 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:30.656230 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee38cb-e781-4d3e-b248-a86ecdc41743 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:30.658175 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bfee38cb-e781-4d3e-b248-a86ecdc41743 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:30.695850 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:30.720611 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee38cb-e781-4d3e-b248-a86ecdc41743 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:30.724530 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf14b0b-c06c-4091-a0ba-cab90764fd1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:30.725175 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf14b0b-c06c-4091-a0ba-cab90764fd1d service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:30.725460 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf14b0b-c06c-4091-a0ba-cab90764fd1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:30.725772 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf14b0b-c06c-4091-a0ba-cab90764fd1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:30.735294 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf14b0b-c06c-4091-a0ba-cab90764fd1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:30.736736 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf14b0b-c06c-4091-a0ba-cab90764fd1d service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:30.746133 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1309/2619] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:11:30 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:11:31.144644 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f45d5-2c63-4f5e-aae0-de7deae29680 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:31.208516 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f45d5-2c63-4f5e-aae0-de7deae29680 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:31.213107 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca589f-b0de-4db3-8461-1d3847e95ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:31.214008 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca589f-b0de-4db3-8461-1d3847e95ed9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:31.214310 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca589f-b0de-4db3-8461-1d3847e95ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:31.214595 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca589f-b0de-4db3-8461-1d3847e95ed9 admin admin] PATH_INFO: `/v3/users/a98a995ea9184c4389b3160d1cc4a95b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:31.215792 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ca589f-b0de-4db3-8461-1d3847e95ed9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a98a995ea9184c4389b3160d1cc4a95b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:31.217764 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ca589f-b0de-4db3-8461-1d3847e95ed9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:31.265307 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f9ca589f-b0de-4db3-8461-1d3847e95ed9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a98a995ea9184c4389b3160d1cc4a95b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:31.265978 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f9ca589f-b0de-4db3-8461-1d3847e95ed9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a98a995ea9184c4389b3160d1cc4a95b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:31.266582 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f9ca589f-b0de-4db3-8461-1d3847e95ed9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a98a995ea9184c4389b3160d1cc4a95b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:31.274045 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f9ca589f-b0de-4db3-8461-1d3847e95ed9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a98a995ea9184c4389b3160d1cc4a95b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:31.281298 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f9ca589f-b0de-4db3-8461-1d3847e95ed9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a98a995ea9184c4389b3160d1cc4a95b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:31.299957 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1311/2620] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:11:31 2022] DELETE /identity/v3/users/a98a995ea9184c4389b3160d1cc4a95b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:31.305368 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cfe197-f292-44f7-a420-a188da67942a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:31.305693 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cfe197-f292-44f7-a420-a188da67942a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:31.305977 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cfe197-f292-44f7-a420-a188da67942a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:31.323284 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c4cfe197-f292-44f7-a420-a188da67942a None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:31.324492 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c4cfe197-f292-44f7-a420-a188da67942a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:31.383469 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1310/2621] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:31.393138 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca589f-b0de-4db3-8461-1d3847e95ed9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:31.394632 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f9ca589f-b0de-4db3-8461-1d3847e95ed9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:31.400013 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:31.454827 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca589f-b0de-4db3-8461-1d3847e95ed9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:31.458653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfab406-1f5b-4d77-81e5-9734cab39ebc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:31.459251 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfab406-1f5b-4d77-81e5-9734cab39ebc service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:31.459537 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfab406-1f5b-4d77-81e5-9734cab39ebc service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:31.459818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfab406-1f5b-4d77-81e5-9734cab39ebc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:31.469622 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfab406-1f5b-4d77-81e5-9734cab39ebc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:31.471052 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfab406-1f5b-4d77-81e5-9734cab39ebc service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:31.481488 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1312/2622] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:11:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:11:31.780012 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cfe197-f292-44f7-a420-a188da67942a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:31.781721 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c4cfe197-f292-44f7-a420-a188da67942a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:31.809743 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cfe197-f292-44f7-a420-a188da67942a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:31.814859 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12da14-6d3d-4590-9ba4-c481ba06b308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:31.816065 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12da14-6d3d-4590-9ba4-c481ba06b308 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:31.816284 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12da14-6d3d-4590-9ba4-c481ba06b308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:31.816571 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12da14-6d3d-4590-9ba4-c481ba06b308 admin admin] PATH_INFO: `/v3/projects/44c8621fee86480a9c21316b3ad98dc3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:31.817801 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c12da14-6d3d-4590-9ba4-c481ba06b308 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44c8621fee86480a9c21316b3ad98dc3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:31.820717 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c12da14-6d3d-4590-9ba4-c481ba06b308 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:31.870691 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5c12da14-6d3d-4590-9ba4-c481ba06b308 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44c8621fee86480a9c21316b3ad98dc3', 'request_id': 'req-5c12da14-6d3d-4590-9ba4-c481ba06b308'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:31.878113 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5c12da14-6d3d-4590-9ba4-c481ba06b308 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44c8621fee86480a9c21316b3ad98dc3', 'request_id': 'req-5c12da14-6d3d-4590-9ba4-c481ba06b308'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:31.884691 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5c12da14-6d3d-4590-9ba4-c481ba06b308 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44c8621fee86480a9c21316b3ad98dc3', 'request_id': 'req-5c12da14-6d3d-4590-9ba4-c481ba06b308'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:31.895704 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5c12da14-6d3d-4590-9ba4-c481ba06b308 admin admin] The token cache is being invalidate because project 44c8621fee86480a9c21316b3ad98dc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:11:31.895997 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5c12da14-6d3d-4590-9ba4-c481ba06b308 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:31.896627 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5c12da14-6d3d-4590-9ba4-c481ba06b308 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:31.898835 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1311/2623] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:11:31 2022] DELETE /identity/v3/projects/44c8621fee86480a9c21316b3ad98dc3 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:31.902831 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfab406-1f5b-4d77-81e5-9734cab39ebc service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:31.912592 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfab406-1f5b-4d77-81e5-9734cab39ebc service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:31.970432 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc53229-cac2-48c9-8720-2f2b4c8b2694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:31.971237 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc53229-cac2-48c9-8720-2f2b4c8b2694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:31.971528 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc53229-cac2-48c9-8720-2f2b4c8b2694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:31.972096 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc53229-cac2-48c9-8720-2f2b4c8b2694 admin admin] PATH_INFO: `/v3/users/8936e4509a044faabfa7e9443b4e10db` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:31.973280 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc53229-cac2-48c9-8720-2f2b4c8b2694 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8936e4509a044faabfa7e9443b4e10db)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:31.975251 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc53229-cac2-48c9-8720-2f2b4c8b2694 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:32.021768 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3fc53229-cac2-48c9-8720-2f2b4c8b2694 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8936e4509a044faabfa7e9443b4e10db'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:32.022420 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3fc53229-cac2-48c9-8720-2f2b4c8b2694 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8936e4509a044faabfa7e9443b4e10db'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:32.023146 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3fc53229-cac2-48c9-8720-2f2b4c8b2694 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8936e4509a044faabfa7e9443b4e10db'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:32.031466 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3fc53229-cac2-48c9-8720-2f2b4c8b2694 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8936e4509a044faabfa7e9443b4e10db'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:32.038619 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3fc53229-cac2-48c9-8720-2f2b4c8b2694 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8936e4509a044faabfa7e9443b4e10db'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:32.058517 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1313/2624] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:11:31 2022] DELETE /identity/v3/users/8936e4509a044faabfa7e9443b4e10db => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:32.335109 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12da14-6d3d-4590-9ba4-c481ba06b308 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:32.336537 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5c12da14-6d3d-4590-9ba4-c481ba06b308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:32.349808 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12da14-6d3d-4590-9ba4-c481ba06b308 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:32.403180 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:32.410110 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257ac99-2333-40e5-a917-6978fb00f500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:32.411051 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257ac99-2333-40e5-a917-6978fb00f500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:32.411528 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257ac99-2333-40e5-a917-6978fb00f500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:32.411967 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257ac99-2333-40e5-a917-6978fb00f500 admin admin] PATH_INFO: `/v3/projects/d0638371ee0441ccb0b5803a4ac41e27` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:32.413289 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5257ac99-2333-40e5-a917-6978fb00f500 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0638371ee0441ccb0b5803a4ac41e27)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:32.415284 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5257ac99-2333-40e5-a917-6978fb00f500 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:32.460963 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5257ac99-2333-40e5-a917-6978fb00f500 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0638371ee0441ccb0b5803a4ac41e27', 'request_id': 'req-5257ac99-2333-40e5-a917-6978fb00f500'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:32.467917 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5257ac99-2333-40e5-a917-6978fb00f500 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0638371ee0441ccb0b5803a4ac41e27', 'request_id': 'req-5257ac99-2333-40e5-a917-6978fb00f500'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:32.473828 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5257ac99-2333-40e5-a917-6978fb00f500 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0638371ee0441ccb0b5803a4ac41e27', 'request_id': 'req-5257ac99-2333-40e5-a917-6978fb00f500'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:32.484799 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5257ac99-2333-40e5-a917-6978fb00f500 admin admin] The token cache is being invalidate because project d0638371ee0441ccb0b5803a4ac41e27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:11:32.485119 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5257ac99-2333-40e5-a917-6978fb00f500 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:32.485791 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5257ac99-2333-40e5-a917-6978fb00f500 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:32.487313 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1312/2625] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:11:32 2022] DELETE /identity/v3/projects/d0638371ee0441ccb0b5803a4ac41e27 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:32.492932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ce13f-2255-4c8f-90ae-74bec2e7ef0d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:32.493394 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ce13f-2255-4c8f-90ae-74bec2e7ef0d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:32.493503 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ce13f-2255-4c8f-90ae-74bec2e7ef0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:32.511782 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-178ce13f-2255-4c8f-90ae-74bec2e7ef0d None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:32.512992 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-178ce13f-2255-4c8f-90ae-74bec2e7ef0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:32.531091 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:32.573091 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1314/2626] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:11:32.580810 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2a023-a22c-4bad-9c0b-a415a75e7675 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:32.581092 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2a023-a22c-4bad-9c0b-a415a75e7675 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:32.581365 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2a023-a22c-4bad-9c0b-a415a75e7675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:32.599525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e6e2a023-a22c-4bad-9c0b-a415a75e7675 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:32.600719 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e6e2a023-a22c-4bad-9c0b-a415a75e7675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:32.609124 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1313/2627] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:32.613120 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ce13f-2255-4c8f-90ae-74bec2e7ef0d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:32.623500 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ce13f-2255-4c8f-90ae-74bec2e7ef0d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:32.627967 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2d227-f0e8-4d97-8b17-e2e4e544ccca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:32.628620 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2d227-f0e8-4d97-8b17-e2e4e544ccca admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:32.628902 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2d227-f0e8-4d97-8b17-e2e4e544ccca admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:32.629179 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2d227-f0e8-4d97-8b17-e2e4e544ccca admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:32.629941 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab2d227-f0e8-4d97-8b17-e2e4e544ccca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:11:32.630277 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab2d227-f0e8-4d97-8b17-e2e4e544ccca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:32.631932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab2d227-f0e8-4d97-8b17-e2e4e544ccca admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:32.643221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1315/2628] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:11:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:11:32.647185 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e2a023-a22c-4bad-9c0b-a415a75e7675 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:32.656269 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e2a023-a22c-4bad-9c0b-a415a75e7675 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:32.660912 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3f1d8-d89e-4d29-bab3-251a44d30e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:32.661542 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3f1d8-d89e-4d29-bab3-251a44d30e1d admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:32.661856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3f1d8-d89e-4d29-bab3-251a44d30e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:32.662130 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3f1d8-d89e-4d29-bab3-251a44d30e1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:32.662810 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3f1d8-d89e-4d29-bab3-251a44d30e1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:32.665254 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3f1d8-d89e-4d29-bab3-251a44d30e1d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:32.691170 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1314/2629] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:11:32 2022] POST /identity/v3/projects => generated 353 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:11:32.694537 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2d227-f0e8-4d97-8b17-e2e4e544ccca admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:32.704578 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2d227-f0e8-4d97-8b17-e2e4e544ccca admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:32.759819 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df77a043-6f64-401b-9ed2-4e34cd708424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:32.760413 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77a043-6f64-401b-9ed2-4e34cd708424 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:32.760700 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77a043-6f64-401b-9ed2-4e34cd708424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:32.760984 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77a043-6f64-401b-9ed2-4e34cd708424 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:32.761722 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df77a043-6f64-401b-9ed2-4e34cd708424 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:32.763635 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df77a043-6f64-401b-9ed2-4e34cd708424 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:32.783214 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1316/2630] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:11:32 2022] POST /identity/v3/users => generated 277 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:11:32.787231 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3f1d8-d89e-4d29-bab3-251a44d30e1d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:32.797539 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3f1d8-d89e-4d29-bab3-251a44d30e1d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:32.802178 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748a13e-537a-4d00-b6a2-405501c3c14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:32.802828 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748a13e-537a-4d00-b6a2-405501c3c14e admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:32.803136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748a13e-537a-4d00-b6a2-405501c3c14e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:32.803390 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748a13e-537a-4d00-b6a2-405501c3c14e admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:32.804115 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b748a13e-537a-4d00-b6a2-405501c3c14e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:32.805735 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b748a13e-537a-4d00-b6a2-405501c3c14e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:32.816822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1315/2631] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:11:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:11:32.821243 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df77a043-6f64-401b-9ed2-4e34cd708424 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:32.831101 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df77a043-6f64-401b-9ed2-4e34cd708424 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:32.836243 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd1fc0-4d89-4ef0-ada1-946b54ada0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:32.836894 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd1fc0-4d89-4ef0-ada1-946b54ada0be admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:32.837181 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd1fc0-4d89-4ef0-ada1-946b54ada0be admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:32.837465 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd1fc0-4d89-4ef0-ada1-946b54ada0be admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:32.838235 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fd1fc0-4d89-4ef0-ada1-946b54ada0be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:32.839831 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fd1fc0-4d89-4ef0-ada1-946b54ada0be admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:32.851804 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1317/2632] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:11:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:11:32.853733 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748a13e-537a-4d00-b6a2-405501c3c14e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:32.863701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748a13e-537a-4d00-b6a2-405501c3c14e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:32.868711 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79080d13-1067-40b4-80ea-43ab0f0c2804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:32.869635 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79080d13-1067-40b4-80ea-43ab0f0c2804 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:32.869921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79080d13-1067-40b4-80ea-43ab0f0c2804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:32.870191 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79080d13-1067-40b4-80ea-43ab0f0c2804 admin admin] PATH_INFO: `/v3/projects/64d6058572c34856b0574800eaa78637/users/e0bf63a3444f4c858217d1c334e8928f/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:32.885811 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79080d13-1067-40b4-80ea-43ab0f0c2804 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64d6058572c34856b0574800eaa78637, user_id=e0bf63a3444f4c858217d1c334e8928f, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:32.888452 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79080d13-1067-40b4-80ea-43ab0f0c2804 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:32.901884 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1316/2633] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:11:32 2022] PUT /identity/v3/projects/64d6058572c34856b0574800eaa78637/users/e0bf63a3444f4c858217d1c334e8928f/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:32.913029 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea08780-839a-4943-bde1-3db095ce1647 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:32.913385 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea08780-839a-4943-bde1-3db095ce1647 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:32.913784 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea08780-839a-4943-bde1-3db095ce1647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:32.944511 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7ea08780-839a-4943-bde1-3db095ce1647 None None] MFA Rules not processed for user `e0bf63a3444f4c858217d1c334e8928f`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:32.946219 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7ea08780-839a-4943-bde1-3db095ce1647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:33.017043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1318/2634] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:32 2022] POST /identity/v3/auth/tokens => generated 3220 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:33.021446 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79080d13-1067-40b4-80ea-43ab0f0c2804 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:33.086658 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79080d13-1067-40b4-80ea-43ab0f0c2804 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:33.091406 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a25be00-d091-4de7-b0c5-8278d7996df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:33.092064 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25be00-d091-4de7-b0c5-8278d7996df2 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:33.092354 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25be00-d091-4de7-b0c5-8278d7996df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:33.092637 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25be00-d091-4de7-b0c5-8278d7996df2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:33.093341 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a25be00-d091-4de7-b0c5-8278d7996df2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:33.095306 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a25be00-d091-4de7-b0c5-8278d7996df2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:33.120638 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1317/2635] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:11:33 2022] POST /identity/v3/projects => generated 357 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:11:33.124695 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea08780-839a-4943-bde1-3db095ce1647 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:33.135553 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea08780-839a-4943-bde1-3db095ce1647 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:33.152322 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:33.196459 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49823555-f580-4764-ba8a-d0ebb68dffdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:33.197155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49823555-f580-4764-ba8a-d0ebb68dffdf admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:33.197443 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49823555-f580-4764-ba8a-d0ebb68dffdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:33.197744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49823555-f580-4764-ba8a-d0ebb68dffdf admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:33.198462 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49823555-f580-4764-ba8a-d0ebb68dffdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:33.200415 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49823555-f580-4764-ba8a-d0ebb68dffdf admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:33.219747 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1319/2636] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:11:33 2022] POST /identity/v3/users => generated 285 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:11:33.224381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a25be00-d091-4de7-b0c5-8278d7996df2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:33.235472 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a25be00-d091-4de7-b0c5-8278d7996df2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:33.246144 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d403fcc-9bb9-4360-9edf-4f3c249a0186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:33.246835 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d403fcc-9bb9-4360-9edf-4f3c249a0186 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:33.247140 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d403fcc-9bb9-4360-9edf-4f3c249a0186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:33.247405 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d403fcc-9bb9-4360-9edf-4f3c249a0186 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:33.248146 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d403fcc-9bb9-4360-9edf-4f3c249a0186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:33.249865 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d403fcc-9bb9-4360-9edf-4f3c249a0186 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:33.252464 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:33.263333 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1318/2637] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:11:33.267196 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49823555-f580-4764-ba8a-d0ebb68dffdf admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:33.278005 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49823555-f580-4764-ba8a-d0ebb68dffdf admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:33.282692 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff6322-dc5d-4d98-bc5b-f003c0d91775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:33.283681 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff6322-dc5d-4d98-bc5b-f003c0d91775 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:33.283971 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff6322-dc5d-4d98-bc5b-f003c0d91775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:33.284251 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff6322-dc5d-4d98-bc5b-f003c0d91775 admin admin] PATH_INFO: `/v3/projects/adda6349bec6418f82f02a623081688d/users/d70f2cbe28ca4953bbc81ba1b06645f0/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:33.299520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ff6322-dc5d-4d98-bc5b-f003c0d91775 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adda6349bec6418f82f02a623081688d, user_id=d70f2cbe28ca4953bbc81ba1b06645f0, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:33.302205 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ff6322-dc5d-4d98-bc5b-f003c0d91775 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:33.314949 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1320/2638] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:11:33 2022] PUT /identity/v3/projects/adda6349bec6418f82f02a623081688d/users/d70f2cbe28ca4953bbc81ba1b06645f0/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:33.318930 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d403fcc-9bb9-4360-9edf-4f3c249a0186 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:33.329458 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d403fcc-9bb9-4360-9edf-4f3c249a0186 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:33.389617 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e832249-16eb-4a03-ae2d-bfaa8d6b9dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:33.390291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e832249-16eb-4a03-ae2d-bfaa8d6b9dc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:33.390572 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e832249-16eb-4a03-ae2d-bfaa8d6b9dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:33.390847 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e832249-16eb-4a03-ae2d-bfaa8d6b9dc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:33.391567 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e832249-16eb-4a03-ae2d-bfaa8d6b9dc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:33.393163 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e832249-16eb-4a03-ae2d-bfaa8d6b9dc5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:33.403650 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1319/2639] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 16:11:33.407829 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff6322-dc5d-4d98-bc5b-f003c0d91775 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:33.417943 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff6322-dc5d-4d98-bc5b-f003c0d91775 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:33.422587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd68182c-215a-420e-a915-433e394ab1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:33.423454 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68182c-215a-420e-a915-433e394ab1b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:33.423735 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68182c-215a-420e-a915-433e394ab1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:33.424009 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68182c-215a-420e-a915-433e394ab1b9 admin admin] PATH_INFO: `/v3/projects/adda6349bec6418f82f02a623081688d/users/d70f2cbe28ca4953bbc81ba1b06645f0/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:33.426156 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd68182c-215a-420e-a915-433e394ab1b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adda6349bec6418f82f02a623081688d, user_id=d70f2cbe28ca4953bbc81ba1b06645f0, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:33.428775 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd68182c-215a-420e-a915-433e394ab1b9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:33.438747 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1321/2640] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:11:33 2022] PUT /identity/v3/projects/adda6349bec6418f82f02a623081688d/users/d70f2cbe28ca4953bbc81ba1b06645f0/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:33.449264 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa46e3-586e-43de-83de-ba51eedd7733 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:33.449555 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa46e3-586e-43de-83de-ba51eedd7733 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:33.449859 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa46e3-586e-43de-83de-ba51eedd7733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:33.476375 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d3aa46e3-586e-43de-83de-ba51eedd7733 None None] MFA Rules not processed for user `d70f2cbe28ca4953bbc81ba1b06645f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:33.477615 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d3aa46e3-586e-43de-83de-ba51eedd7733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:33.544161 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1320/2641] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:33 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:33.555031 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd68182c-215a-420e-a915-433e394ab1b9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:33.556754 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-dd68182c-215a-420e-a915-433e394ab1b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:33.614429 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd68182c-215a-420e-a915-433e394ab1b9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:33.618674 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d4e015-b60f-46dd-a016-f7f11a1f8c31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:33.619414 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d4e015-b60f-46dd-a016-f7f11a1f8c31 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:33.619704 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d4e015-b60f-46dd-a016-f7f11a1f8c31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:33.619981 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d4e015-b60f-46dd-a016-f7f11a1f8c31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:33.629209 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d4e015-b60f-46dd-a016-f7f11a1f8c31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:33.630733 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d4e015-b60f-46dd-a016-f7f11a1f8c31 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:33.641167 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1322/2642] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:11:33 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:11:33.712237 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aa46e3-586e-43de-83de-ba51eedd7733 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:33.722363 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aa46e3-586e-43de-83de-ba51eedd7733 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:33.726938 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a4d9ab-3cb5-437b-9def-6b3789e633d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:33.727724 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4d9ab-3cb5-437b-9def-6b3789e633d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:33.728015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4d9ab-3cb5-437b-9def-6b3789e633d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:33.728304 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4d9ab-3cb5-437b-9def-6b3789e633d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:33.783982 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a4d9ab-3cb5-437b-9def-6b3789e633d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:33.785522 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a4d9ab-3cb5-437b-9def-6b3789e633d1 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:33.795748 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1321/2643] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:11:33 2022] GET /identity/v3/auth/tokens => generated 3220 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:11:34.191587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d4e015-b60f-46dd-a016-f7f11a1f8c31 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:34.192977 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-03d4e015-b60f-46dd-a016-f7f11a1f8c31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:34.259178 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d4e015-b60f-46dd-a016-f7f11a1f8c31 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:34.263693 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d711b954-c395-4cf2-8278-0c32b71595fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:34.264561 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711b954-c395-4cf2-8278-0c32b71595fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:34.264848 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711b954-c395-4cf2-8278-0c32b71595fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:34.265126 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711b954-c395-4cf2-8278-0c32b71595fc admin admin] PATH_INFO: `/v3/users/479293e2b0894a5e8dd99f56a811a14f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:34.266333 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d711b954-c395-4cf2-8278-0c32b71595fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=479293e2b0894a5e8dd99f56a811a14f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:34.268246 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d711b954-c395-4cf2-8278-0c32b71595fc admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:34.283856 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:34.321533 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d711b954-c395-4cf2-8278-0c32b71595fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '479293e2b0894a5e8dd99f56a811a14f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:34.322138 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d711b954-c395-4cf2-8278-0c32b71595fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '479293e2b0894a5e8dd99f56a811a14f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:34.322769 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d711b954-c395-4cf2-8278-0c32b71595fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '479293e2b0894a5e8dd99f56a811a14f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:34.330105 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d711b954-c395-4cf2-8278-0c32b71595fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '479293e2b0894a5e8dd99f56a811a14f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:34.338171 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d711b954-c395-4cf2-8278-0c32b71595fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '479293e2b0894a5e8dd99f56a811a14f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:34.357772 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1323/2644] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:11:34 2022] DELETE /identity/v3/users/479293e2b0894a5e8dd99f56a811a14f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:34.362353 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5cf99-25c4-462d-973e-082b45305181 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:34.362646 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5cf99-25c4-462d-973e-082b45305181 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:34.362924 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5cf99-25c4-462d-973e-082b45305181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:34.379598 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5bb5cf99-25c4-462d-973e-082b45305181 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:34.380799 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5bb5cf99-25c4-462d-973e-082b45305181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:34.393710 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:34.441248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1322/2645] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:34.451589 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d711b954-c395-4cf2-8278-0c32b71595fc admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:34.452991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d711b954-c395-4cf2-8278-0c32b71595fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:34.511025 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d711b954-c395-4cf2-8278-0c32b71595fc admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:34.514653 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e95bf3-f57e-4039-a478-55ef2450c732 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:34.515291 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e95bf3-f57e-4039-a478-55ef2450c732 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:34.515575 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e95bf3-f57e-4039-a478-55ef2450c732 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:34.515851 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e95bf3-f57e-4039-a478-55ef2450c732 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:34.524808 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e95bf3-f57e-4039-a478-55ef2450c732 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:34.526265 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e95bf3-f57e-4039-a478-55ef2450c732 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:34.535726 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1324/2646] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:11:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:11:34.806225 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5cf99-25c4-462d-973e-082b45305181 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:34.807694 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5bb5cf99-25c4-462d-973e-082b45305181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:34.821270 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5cf99-25c4-462d-973e-082b45305181 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:34.825964 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15f84e-1ac3-4529-80c1-c258117d6a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:34.826833 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15f84e-1ac3-4529-80c1-c258117d6a55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:34.827145 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15f84e-1ac3-4529-80c1-c258117d6a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:34.827541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15f84e-1ac3-4529-80c1-c258117d6a55 admin admin] PATH_INFO: `/v3/projects/16826e7654104049808d2cf7fe78fbda` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:34.828743 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af15f84e-1ac3-4529-80c1-c258117d6a55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16826e7654104049808d2cf7fe78fbda)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:34.830752 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af15f84e-1ac3-4529-80c1-c258117d6a55 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:34.876922 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-af15f84e-1ac3-4529-80c1-c258117d6a55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16826e7654104049808d2cf7fe78fbda', 'request_id': 'req-af15f84e-1ac3-4529-80c1-c258117d6a55'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:34.884335 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-af15f84e-1ac3-4529-80c1-c258117d6a55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16826e7654104049808d2cf7fe78fbda', 'request_id': 'req-af15f84e-1ac3-4529-80c1-c258117d6a55'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:34.889910 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-af15f84e-1ac3-4529-80c1-c258117d6a55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16826e7654104049808d2cf7fe78fbda', 'request_id': 'req-af15f84e-1ac3-4529-80c1-c258117d6a55'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:34.901317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-af15f84e-1ac3-4529-80c1-c258117d6a55 admin admin] The token cache is being invalidate because project 16826e7654104049808d2cf7fe78fbda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:11:34.901630 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-af15f84e-1ac3-4529-80c1-c258117d6a55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:34.902263 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-af15f84e-1ac3-4529-80c1-c258117d6a55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:34.903875 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1323/2647] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:11:34 2022] DELETE /identity/v3/projects/16826e7654104049808d2cf7fe78fbda => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:34.909683 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9d100-54b0-4ff3-b8fa-ac8513da4b67 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:34.910222 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9d100-54b0-4ff3-b8fa-ac8513da4b67 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:34.910364 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9d100-54b0-4ff3-b8fa-ac8513da4b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:34.927227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-64f9d100-54b0-4ff3-b8fa-ac8513da4b67 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:34.928432 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-64f9d100-54b0-4ff3-b8fa-ac8513da4b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:34.990493 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1325/2648] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:34.998932 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe245d02-f632-4cfb-aac2-83bb4b3b5940 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:34.999298 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe245d02-f632-4cfb-aac2-83bb4b3b5940 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:34.999748 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe245d02-f632-4cfb-aac2-83bb4b3b5940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:35.020196 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fe245d02-f632-4cfb-aac2-83bb4b3b5940 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:35.021423 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fe245d02-f632-4cfb-aac2-83bb4b3b5940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:35.029488 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1324/2649] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:35.033830 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9d100-54b0-4ff3-b8fa-ac8513da4b67 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:35.049618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9d100-54b0-4ff3-b8fa-ac8513da4b67 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:35.055594 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9582f0a6-4472-45e7-ac7d-44e0bbc23942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:35.056276 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582f0a6-4472-45e7-ac7d-44e0bbc23942 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:35.056565 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582f0a6-4472-45e7-ac7d-44e0bbc23942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:35.056846 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582f0a6-4472-45e7-ac7d-44e0bbc23942 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:35.057602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9582f0a6-4472-45e7-ac7d-44e0bbc23942 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:11:35.057926 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9582f0a6-4472-45e7-ac7d-44e0bbc23942 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:35.059528 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9582f0a6-4472-45e7-ac7d-44e0bbc23942 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:35.075530 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1326/2650] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:11:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 16:11:35.077063 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe245d02-f632-4cfb-aac2-83bb4b3b5940 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:35.088135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe245d02-f632-4cfb-aac2-83bb4b3b5940 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:35.092837 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f71c90-cd44-4536-a773-859acb6a52e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:35.093516 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f71c90-cd44-4536-a773-859acb6a52e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:35.093841 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f71c90-cd44-4536-a773-859acb6a52e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:35.094120 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f71c90-cd44-4536-a773-859acb6a52e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:35.094839 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f71c90-cd44-4536-a773-859acb6a52e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:35.097087 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f71c90-cd44-4536-a773-859acb6a52e2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:35.122370 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1325/2651] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:11:35 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:11:35.126574 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9582f0a6-4472-45e7-ac7d-44e0bbc23942 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:35.137094 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9582f0a6-4472-45e7-ac7d-44e0bbc23942 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:35.157850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:35.195998 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f2906-656b-47f0-b6d8-8e1e3dd17f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:35.196661 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f2906-656b-47f0-b6d8-8e1e3dd17f47 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:35.196945 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f2906-656b-47f0-b6d8-8e1e3dd17f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:35.197222 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f2906-656b-47f0-b6d8-8e1e3dd17f47 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:35.197945 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f2906-656b-47f0-b6d8-8e1e3dd17f47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:35.199858 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f2906-656b-47f0-b6d8-8e1e3dd17f47 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:35.220090 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1327/2652] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:11:35 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:11:35.224622 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f71c90-cd44-4536-a773-859acb6a52e2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:35.235535 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f71c90-cd44-4536-a773-859acb6a52e2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:35.240202 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe8a524-dbc0-46a4-881e-5821efbf0ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:35.240865 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8a524-dbc0-46a4-881e-5821efbf0ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:35.241149 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8a524-dbc0-46a4-881e-5821efbf0ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:35.241429 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8a524-dbc0-46a4-881e-5821efbf0ed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:35.242195 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe8a524-dbc0-46a4-881e-5821efbf0ed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:35.243801 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe8a524-dbc0-46a4-881e-5821efbf0ed1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:35.254395 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1326/2653] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:11:35.258536 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f2906-656b-47f0-b6d8-8e1e3dd17f47 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:35.268634 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f2906-656b-47f0-b6d8-8e1e3dd17f47 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:35.273907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56bf164-8502-4a34-bfea-2acb4a2a7bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:35.274585 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bf164-8502-4a34-bfea-2acb4a2a7bea admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:35.274871 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bf164-8502-4a34-bfea-2acb4a2a7bea admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:35.275150 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bf164-8502-4a34-bfea-2acb4a2a7bea admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:35.275878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56bf164-8502-4a34-bfea-2acb4a2a7bea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:35.277490 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56bf164-8502-4a34-bfea-2acb4a2a7bea admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:35.287450 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1328/2654] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:11:35.292450 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe8a524-dbc0-46a4-881e-5821efbf0ed1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:35.302483 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe8a524-dbc0-46a4-881e-5821efbf0ed1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:35.307133 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e22a8-74d7-4d8b-b00e-9e2e582ad540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:35.307997 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e22a8-74d7-4d8b-b00e-9e2e582ad540 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:35.308283 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e22a8-74d7-4d8b-b00e-9e2e582ad540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:35.308562 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e22a8-74d7-4d8b-b00e-9e2e582ad540 admin admin] PATH_INFO: `/v3/projects/f77a497c3c4e495cbe0bccbc3e1163cc/users/e92eff99aa9f4a70afe275d698760f67/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:35.323793 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841e22a8-74d7-4d8b-b00e-9e2e582ad540 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f77a497c3c4e495cbe0bccbc3e1163cc, user_id=e92eff99aa9f4a70afe275d698760f67, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:35.326462 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841e22a8-74d7-4d8b-b00e-9e2e582ad540 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:35.339700 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1327/2655] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:11:35 2022] PUT /identity/v3/projects/f77a497c3c4e495cbe0bccbc3e1163cc/users/e92eff99aa9f4a70afe275d698760f67/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:11:35.350034 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c915c9-92f2-4cde-afb6-ef404cbeaf48 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:35.350327 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c915c9-92f2-4cde-afb6-ef404cbeaf48 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:35.350608 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c915c9-92f2-4cde-afb6-ef404cbeaf48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:35.376717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-11c915c9-92f2-4cde-afb6-ef404cbeaf48 None None] MFA Rules not processed for user `e92eff99aa9f4a70afe275d698760f67`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:35.377944 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-11c915c9-92f2-4cde-afb6-ef404cbeaf48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:35.444293 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1329/2656] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:35.455241 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e22a8-74d7-4d8b-b00e-9e2e582ad540 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:35.456939 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-841e22a8-74d7-4d8b-b00e-9e2e582ad540 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:35.500317 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:35.517864 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e22a8-74d7-4d8b-b00e-9e2e582ad540 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:35.521475 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b99ae6-443e-407e-bb1c-0687679f3109 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:35.522154 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b99ae6-443e-407e-bb1c-0687679f3109 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:35.522439 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b99ae6-443e-407e-bb1c-0687679f3109 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:35.522717 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b99ae6-443e-407e-bb1c-0687679f3109 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:35.531480 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b99ae6-443e-407e-bb1c-0687679f3109 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:35.532920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b99ae6-443e-407e-bb1c-0687679f3109 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:35.541866 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1328/2657] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:11:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:11:38.495151 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c915c9-92f2-4cde-afb6-ef404cbeaf48 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:38.555595 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c915c9-92f2-4cde-afb6-ef404cbeaf48 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:38.560145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b381b9-6d9c-4d83-a846-fdb2aa80d459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:38.561016 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b381b9-6d9c-4d83-a846-fdb2aa80d459 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:38.561298 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b381b9-6d9c-4d83-a846-fdb2aa80d459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:38.561603 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b381b9-6d9c-4d83-a846-fdb2aa80d459 admin admin] PATH_INFO: `/v3/users/e92eff99aa9f4a70afe275d698760f67` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:38.562762 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b381b9-6d9c-4d83-a846-fdb2aa80d459 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e92eff99aa9f4a70afe275d698760f67)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:38.564667 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b381b9-6d9c-4d83-a846-fdb2aa80d459 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:38.602578 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:38.613609 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-22b381b9-6d9c-4d83-a846-fdb2aa80d459 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e92eff99aa9f4a70afe275d698760f67'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:38.614505 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-22b381b9-6d9c-4d83-a846-fdb2aa80d459 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e92eff99aa9f4a70afe275d698760f67'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:38.615130 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-22b381b9-6d9c-4d83-a846-fdb2aa80d459 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e92eff99aa9f4a70afe275d698760f67'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:38.622727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-22b381b9-6d9c-4d83-a846-fdb2aa80d459 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e92eff99aa9f4a70afe275d698760f67'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:38.629896 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-22b381b9-6d9c-4d83-a846-fdb2aa80d459 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e92eff99aa9f4a70afe275d698760f67'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:38.648456 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1330/2658] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:11:38 2022] DELETE /identity/v3/users/e92eff99aa9f4a70afe275d698760f67 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:38.653121 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dba114-88c3-436c-bd66-270e4baf2ad6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:38.653679 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dba114-88c3-436c-bd66-270e4baf2ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:38.653781 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dba114-88c3-436c-bd66-270e4baf2ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:38.670635 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-14dba114-88c3-436c-bd66-270e4baf2ad6 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:38.671833 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-14dba114-88c3-436c-bd66-270e4baf2ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:38.733649 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1329/2659] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:38.743397 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b381b9-6d9c-4d83-a846-fdb2aa80d459 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:38.744793 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-22b381b9-6d9c-4d83-a846-fdb2aa80d459 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:38.803443 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b381b9-6d9c-4d83-a846-fdb2aa80d459 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:38.807181 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d651637-0d8f-4cbb-b869-7842c8f4bafd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:38.807863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d651637-0d8f-4cbb-b869-7842c8f4bafd service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:38.808177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d651637-0d8f-4cbb-b869-7842c8f4bafd service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:38.808492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d651637-0d8f-4cbb-b869-7842c8f4bafd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:38.817649 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d651637-0d8f-4cbb-b869-7842c8f4bafd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:38.819097 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d651637-0d8f-4cbb-b869-7842c8f4bafd service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:38.829230 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1331/2660] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:11:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 16:11:39.112620 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dba114-88c3-436c-bd66-270e4baf2ad6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:39.114562 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-14dba114-88c3-436c-bd66-270e4baf2ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:39.127887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dba114-88c3-436c-bd66-270e4baf2ad6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:39.134503 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced359a5-2e38-46ee-8f78-c106cd162b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:39.135440 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced359a5-2e38-46ee-8f78-c106cd162b0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:39.135725 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced359a5-2e38-46ee-8f78-c106cd162b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:39.136006 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced359a5-2e38-46ee-8f78-c106cd162b0e admin admin] PATH_INFO: `/v3/projects/f77a497c3c4e495cbe0bccbc3e1163cc` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:39.137336 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced359a5-2e38-46ee-8f78-c106cd162b0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f77a497c3c4e495cbe0bccbc3e1163cc)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:39.139302 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced359a5-2e38-46ee-8f78-c106cd162b0e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:39.190584 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ced359a5-2e38-46ee-8f78-c106cd162b0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f77a497c3c4e495cbe0bccbc3e1163cc', 'request_id': 'req-ced359a5-2e38-46ee-8f78-c106cd162b0e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:39.198862 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ced359a5-2e38-46ee-8f78-c106cd162b0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f77a497c3c4e495cbe0bccbc3e1163cc', 'request_id': 'req-ced359a5-2e38-46ee-8f78-c106cd162b0e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:39.204472 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ced359a5-2e38-46ee-8f78-c106cd162b0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f77a497c3c4e495cbe0bccbc3e1163cc', 'request_id': 'req-ced359a5-2e38-46ee-8f78-c106cd162b0e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:39.215983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ced359a5-2e38-46ee-8f78-c106cd162b0e admin admin] The token cache is being invalidate because project f77a497c3c4e495cbe0bccbc3e1163cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:11:39.216744 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ced359a5-2e38-46ee-8f78-c106cd162b0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:39.217390 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ced359a5-2e38-46ee-8f78-c106cd162b0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:39.218988 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1330/2661] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:11:39 2022] DELETE /identity/v3/projects/f77a497c3c4e495cbe0bccbc3e1163cc => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:39.224935 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee86c90-b0a7-4455-9f2d-9e7640157062 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:39.225717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee86c90-b0a7-4455-9f2d-9e7640157062 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:39.225717 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee86c90-b0a7-4455-9f2d-9e7640157062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:39.244633 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2ee86c90-b0a7-4455-9f2d-9e7640157062 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:39.245952 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2ee86c90-b0a7-4455-9f2d-9e7640157062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:39.305751 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1332/2662] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:11:39.313499 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07312e13-f0be-4d61-9e38-a5b32574258a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:39.313822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07312e13-f0be-4d61-9e38-a5b32574258a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:39.314102 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07312e13-f0be-4d61-9e38-a5b32574258a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:39.332929 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-07312e13-f0be-4d61-9e38-a5b32574258a None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:39.334172 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-07312e13-f0be-4d61-9e38-a5b32574258a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:39.343024 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1331/2663] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:39.347331 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee86c90-b0a7-4455-9f2d-9e7640157062 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:39.357365 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee86c90-b0a7-4455-9f2d-9e7640157062 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:39.362009 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9aa47-6653-47bd-b2b9-f257b5f352e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:39.362690 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9aa47-6653-47bd-b2b9-f257b5f352e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:39.363010 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9aa47-6653-47bd-b2b9-f257b5f352e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:39.363316 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9aa47-6653-47bd-b2b9-f257b5f352e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:39.364092 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c9aa47-6653-47bd-b2b9-f257b5f352e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:11:39.364429 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c9aa47-6653-47bd-b2b9-f257b5f352e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:39.366079 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c9aa47-6653-47bd-b2b9-f257b5f352e9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:39.378694 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1333/2664] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:11:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 16:11:39.383480 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07312e13-f0be-4d61-9e38-a5b32574258a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:39.392826 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07312e13-f0be-4d61-9e38-a5b32574258a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:39.397450 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70493d-caa2-47ed-aac5-a12cb42d1cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:39.398754 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70493d-caa2-47ed-aac5-a12cb42d1cdb admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:39.399035 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70493d-caa2-47ed-aac5-a12cb42d1cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:39.399309 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70493d-caa2-47ed-aac5-a12cb42d1cdb admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:39.399996 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70493d-caa2-47ed-aac5-a12cb42d1cdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:39.401930 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70493d-caa2-47ed-aac5-a12cb42d1cdb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:39.426993 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1332/2665] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:11:39 2022] POST /identity/v3/projects => generated 365 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:11:39.431104 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9aa47-6653-47bd-b2b9-f257b5f352e9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:39.441403 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9aa47-6653-47bd-b2b9-f257b5f352e9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:39.483262 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:39.500641 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44625d6-3add-4623-86eb-2a2e518f70cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:39.501307 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44625d6-3add-4623-86eb-2a2e518f70cd admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:39.501559 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44625d6-3add-4623-86eb-2a2e518f70cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:39.501865 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44625d6-3add-4623-86eb-2a2e518f70cd admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:39.502554 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44625d6-3add-4623-86eb-2a2e518f70cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:39.504466 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44625d6-3add-4623-86eb-2a2e518f70cd admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:39.524847 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1334/2666] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:11:39 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:11:39.528867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70493d-caa2-47ed-aac5-a12cb42d1cdb admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:39.539325 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70493d-caa2-47ed-aac5-a12cb42d1cdb admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:39.544073 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d4f05-4e97-4b05-ba60-624913d4ca5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:39.544730 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d4f05-4e97-4b05-ba60-624913d4ca5e admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:39.545016 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d4f05-4e97-4b05-ba60-624913d4ca5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:39.545293 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d4f05-4e97-4b05-ba60-624913d4ca5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:39.546060 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d4f05-4e97-4b05-ba60-624913d4ca5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:39.547670 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d4f05-4e97-4b05-ba60-624913d4ca5e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:39.557998 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1333/2667] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:11:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:11:39.561702 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44625d6-3add-4623-86eb-2a2e518f70cd admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:39.571991 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44625d6-3add-4623-86eb-2a2e518f70cd admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:39.576899 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0afcd4-108f-415f-8b34-96b2bff568ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:39.577552 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0afcd4-108f-415f-8b34-96b2bff568ef admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:39.577870 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0afcd4-108f-415f-8b34-96b2bff568ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:39.578281 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0afcd4-108f-415f-8b34-96b2bff568ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:39.579033 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0afcd4-108f-415f-8b34-96b2bff568ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:39.580705 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0afcd4-108f-415f-8b34-96b2bff568ef admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:39.590871 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1335/2668] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:11:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:11:39.594977 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d4f05-4e97-4b05-ba60-624913d4ca5e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:39.606186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d4f05-4e97-4b05-ba60-624913d4ca5e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:39.610758 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9690afa-57fd-4955-815f-56f7337271cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:39.611632 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9690afa-57fd-4955-815f-56f7337271cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:39.611915 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9690afa-57fd-4955-815f-56f7337271cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:39.612192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9690afa-57fd-4955-815f-56f7337271cb admin admin] PATH_INFO: `/v3/projects/3227949ec1064cd0bff5468018ed6bc9/users/12660992e7ec4912b77a82c4f3edc607/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:39.628581 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9690afa-57fd-4955-815f-56f7337271cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3227949ec1064cd0bff5468018ed6bc9, user_id=12660992e7ec4912b77a82c4f3edc607, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:39.631261 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9690afa-57fd-4955-815f-56f7337271cb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:39.639679 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:39.644679 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1334/2669] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:11:39 2022] PUT /identity/v3/projects/3227949ec1064cd0bff5468018ed6bc9/users/12660992e7ec4912b77a82c4f3edc607/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:39.655897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d5171-5fe9-482a-88c3-cb79de3fdc00 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:39.656257 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d5171-5fe9-482a-88c3-cb79de3fdc00 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:39.656544 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d5171-5fe9-482a-88c3-cb79de3fdc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:39.684138 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4f9d5171-5fe9-482a-88c3-cb79de3fdc00 None None] MFA Rules not processed for user `12660992e7ec4912b77a82c4f3edc607`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:39.685345 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4f9d5171-5fe9-482a-88c3-cb79de3fdc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:39.753027 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1336/2670] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:39 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:39.761737 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9690afa-57fd-4955-815f-56f7337271cb admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:39.763424 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a9690afa-57fd-4955-815f-56f7337271cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:39.820371 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9690afa-57fd-4955-815f-56f7337271cb admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:39.823932 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee2049-3129-4309-afb3-2d707b949bb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:39.824576 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee2049-3129-4309-afb3-2d707b949bb6 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:39.824860 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee2049-3129-4309-afb3-2d707b949bb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:39.825136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee2049-3129-4309-afb3-2d707b949bb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:39.833366 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee2049-3129-4309-afb3-2d707b949bb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:39.834845 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee2049-3129-4309-afb3-2d707b949bb6 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:39.844033 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1335/2671] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:11:39 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:11:49.746706 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=47,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:49.840047 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=26 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:50.670769 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9d5171-5fe9-482a-88c3-cb79de3fdc00 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:50.672177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4f9d5171-5fe9-482a-88c3-cb79de3fdc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:50.741121 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9d5171-5fe9-482a-88c3-cb79de3fdc00 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:50.747431 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebfb0d-1a2b-45d7-8d59-94e3712177a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:50.748617 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebfb0d-1a2b-45d7-8d59-94e3712177a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:50.748933 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebfb0d-1a2b-45d7-8d59-94e3712177a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:50.749185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebfb0d-1a2b-45d7-8d59-94e3712177a3 admin admin] PATH_INFO: `/v3/users/e0bf63a3444f4c858217d1c334e8928f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:50.750438 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ebfb0d-1a2b-45d7-8d59-94e3712177a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0bf63a3444f4c858217d1c334e8928f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:50.756384 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ebfb0d-1a2b-45d7-8d59-94e3712177a3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:50.825098 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-72ebfb0d-1a2b-45d7-8d59-94e3712177a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0bf63a3444f4c858217d1c334e8928f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:50.825798 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-72ebfb0d-1a2b-45d7-8d59-94e3712177a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0bf63a3444f4c858217d1c334e8928f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:50.826455 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-72ebfb0d-1a2b-45d7-8d59-94e3712177a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0bf63a3444f4c858217d1c334e8928f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:50.833668 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-72ebfb0d-1a2b-45d7-8d59-94e3712177a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0bf63a3444f4c858217d1c334e8928f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:50.847676 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-72ebfb0d-1a2b-45d7-8d59-94e3712177a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0bf63a3444f4c858217d1c334e8928f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:50.867394 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1337/2672] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:11:50 2022] DELETE /identity/v3/users/e0bf63a3444f4c858217d1c334e8928f => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:11:50.872525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469c1c2-aee4-43c5-ba58-f110f333a1b3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:50.872999 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469c1c2-aee4-43c5-ba58-f110f333a1b3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:50.873346 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469c1c2-aee4-43c5-ba58-f110f333a1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:50.890957 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3469c1c2-aee4-43c5-ba58-f110f333a1b3 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:50.897768 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3469c1c2-aee4-43c5-ba58-f110f333a1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:50.958821 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1336/2673] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:50.969053 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebfb0d-1a2b-45d7-8d59-94e3712177a3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:50.970505 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-72ebfb0d-1a2b-45d7-8d59-94e3712177a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:51.030555 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebfb0d-1a2b-45d7-8d59-94e3712177a3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:51.034248 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88085a2a-e99a-4ae6-a07f-2b7301baa545 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:51.034917 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88085a2a-e99a-4ae6-a07f-2b7301baa545 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:51.035178 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88085a2a-e99a-4ae6-a07f-2b7301baa545 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:51.035475 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88085a2a-e99a-4ae6-a07f-2b7301baa545 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:51.045727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88085a2a-e99a-4ae6-a07f-2b7301baa545 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:51.047146 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88085a2a-e99a-4ae6-a07f-2b7301baa545 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:51.057126 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1338/2674] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:11:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:11:51.391097 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3469c1c2-aee4-43c5-ba58-f110f333a1b3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:51.393180 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3469c1c2-aee4-43c5-ba58-f110f333a1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:51.407025 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3469c1c2-aee4-43c5-ba58-f110f333a1b3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:51.413918 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4a791-07e1-4c66-90c6-9a2ebb2d44b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:51.414792 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4a791-07e1-4c66-90c6-9a2ebb2d44b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:51.415072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4a791-07e1-4c66-90c6-9a2ebb2d44b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:51.415447 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4a791-07e1-4c66-90c6-9a2ebb2d44b7 admin admin] PATH_INFO: `/v3/projects/64d6058572c34856b0574800eaa78637` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:51.416765 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b4a791-07e1-4c66-90c6-9a2ebb2d44b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64d6058572c34856b0574800eaa78637)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:51.418992 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b4a791-07e1-4c66-90c6-9a2ebb2d44b7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:51.480657 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-21b4a791-07e1-4c66-90c6-9a2ebb2d44b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64d6058572c34856b0574800eaa78637', 'request_id': 'req-21b4a791-07e1-4c66-90c6-9a2ebb2d44b7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:51.489312 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-21b4a791-07e1-4c66-90c6-9a2ebb2d44b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64d6058572c34856b0574800eaa78637', 'request_id': 'req-21b4a791-07e1-4c66-90c6-9a2ebb2d44b7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:51.495588 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-21b4a791-07e1-4c66-90c6-9a2ebb2d44b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64d6058572c34856b0574800eaa78637', 'request_id': 'req-21b4a791-07e1-4c66-90c6-9a2ebb2d44b7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:51.508340 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-21b4a791-07e1-4c66-90c6-9a2ebb2d44b7 admin admin] The token cache is being invalidate because project 64d6058572c34856b0574800eaa78637 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:11:51.508655 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-21b4a791-07e1-4c66-90c6-9a2ebb2d44b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:51.509272 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-21b4a791-07e1-4c66-90c6-9a2ebb2d44b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:51.511016 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1337/2675] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:11:51 2022] DELETE /identity/v3/projects/64d6058572c34856b0574800eaa78637 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:51.515461 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88085a2a-e99a-4ae6-a07f-2b7301baa545 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:51.526273 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88085a2a-e99a-4ae6-a07f-2b7301baa545 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:51.543934 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:51.585610 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc07361b-87ee-4753-932e-63d940768d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:51.586636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07361b-87ee-4753-932e-63d940768d58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:51.586923 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07361b-87ee-4753-932e-63d940768d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:51.587203 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07361b-87ee-4753-932e-63d940768d58 admin admin] PATH_INFO: `/v3/users/d70f2cbe28ca4953bbc81ba1b06645f0` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:51.588385 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc07361b-87ee-4753-932e-63d940768d58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d70f2cbe28ca4953bbc81ba1b06645f0)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:51.590339 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc07361b-87ee-4753-932e-63d940768d58 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:51.636553 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fc07361b-87ee-4753-932e-63d940768d58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd70f2cbe28ca4953bbc81ba1b06645f0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:51.637116 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fc07361b-87ee-4753-932e-63d940768d58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd70f2cbe28ca4953bbc81ba1b06645f0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:51.637741 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fc07361b-87ee-4753-932e-63d940768d58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd70f2cbe28ca4953bbc81ba1b06645f0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:51.644673 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fc07361b-87ee-4753-932e-63d940768d58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd70f2cbe28ca4953bbc81ba1b06645f0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:51.653064 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fc07361b-87ee-4753-932e-63d940768d58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd70f2cbe28ca4953bbc81ba1b06645f0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:51.669886 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1339/2676] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:11:51 2022] DELETE /identity/v3/users/d70f2cbe28ca4953bbc81ba1b06645f0 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:51.952584 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4a791-07e1-4c66-90c6-9a2ebb2d44b7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:51.955078 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-21b4a791-07e1-4c66-90c6-9a2ebb2d44b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:51.968666 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4a791-07e1-4c66-90c6-9a2ebb2d44b7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:52.028668 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6392cbe-2f84-4e7a-8329-6911ea5ab911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:52.029523 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6392cbe-2f84-4e7a-8329-6911ea5ab911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:52.029839 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6392cbe-2f84-4e7a-8329-6911ea5ab911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:52.030118 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6392cbe-2f84-4e7a-8329-6911ea5ab911 admin admin] PATH_INFO: `/v3/projects/adda6349bec6418f82f02a623081688d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:52.031288 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6392cbe-2f84-4e7a-8329-6911ea5ab911 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adda6349bec6418f82f02a623081688d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:52.033216 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6392cbe-2f84-4e7a-8329-6911ea5ab911 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:52.062905 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:52.080769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b6392cbe-2f84-4e7a-8329-6911ea5ab911 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adda6349bec6418f82f02a623081688d', 'request_id': 'req-b6392cbe-2f84-4e7a-8329-6911ea5ab911'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:52.088444 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b6392cbe-2f84-4e7a-8329-6911ea5ab911 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adda6349bec6418f82f02a623081688d', 'request_id': 'req-b6392cbe-2f84-4e7a-8329-6911ea5ab911'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:52.094232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b6392cbe-2f84-4e7a-8329-6911ea5ab911 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adda6349bec6418f82f02a623081688d', 'request_id': 'req-b6392cbe-2f84-4e7a-8329-6911ea5ab911'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:52.105566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b6392cbe-2f84-4e7a-8329-6911ea5ab911 admin admin] The token cache is being invalidate because project adda6349bec6418f82f02a623081688d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:11:52.105879 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b6392cbe-2f84-4e7a-8329-6911ea5ab911 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:52.106507 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b6392cbe-2f84-4e7a-8329-6911ea5ab911 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:11:52.108019 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1338/2677] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:11:51 2022] DELETE /identity/v3/projects/adda6349bec6418f82f02a623081688d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:52.114419 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1dcb3-94f7-4ad9-82e5-09b9d3883751 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:52.114914 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1dcb3-94f7-4ad9-82e5-09b9d3883751 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:52.115127 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1dcb3-94f7-4ad9-82e5-09b9d3883751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:52.133447 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-eba1dcb3-94f7-4ad9-82e5-09b9d3883751 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:52.134609 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-eba1dcb3-94f7-4ad9-82e5-09b9d3883751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:52.191357 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1340/2678] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:52.199782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014916b8-f81d-4d3c-a70f-02c33f1b9cbe None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:52.200078 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014916b8-f81d-4d3c-a70f-02c33f1b9cbe None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:52.200363 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014916b8-f81d-4d3c-a70f-02c33f1b9cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:52.219967 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-014916b8-f81d-4d3c-a70f-02c33f1b9cbe None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:52.221201 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-014916b8-f81d-4d3c-a70f-02c33f1b9cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:52.229380 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1339/2679] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:52.233502 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba1dcb3-94f7-4ad9-82e5-09b9d3883751 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:52.243627 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba1dcb3-94f7-4ad9-82e5-09b9d3883751 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:52.248214 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f342c1f-0122-45b7-ac2f-7106c9d407ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:52.248871 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f342c1f-0122-45b7-ac2f-7106c9d407ce admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:52.249174 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f342c1f-0122-45b7-ac2f-7106c9d407ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:52.249455 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f342c1f-0122-45b7-ac2f-7106c9d407ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:52.250243 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f342c1f-0122-45b7-ac2f-7106c9d407ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:11:52.250554 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f342c1f-0122-45b7-ac2f-7106c9d407ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:52.252141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f342c1f-0122-45b7-ac2f-7106c9d407ce admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:52.268261 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1341/2680] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:11:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:11:52.269607 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014916b8-f81d-4d3c-a70f-02c33f1b9cbe None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:52.281394 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014916b8-f81d-4d3c-a70f-02c33f1b9cbe None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:52.287148 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6834ec7-952f-4c89-b194-aa6f61a69ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:52.288659 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6834ec7-952f-4c89-b194-aa6f61a69ff8 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:52.289050 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6834ec7-952f-4c89-b194-aa6f61a69ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:52.289546 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6834ec7-952f-4c89-b194-aa6f61a69ff8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:52.290745 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6834ec7-952f-4c89-b194-aa6f61a69ff8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:52.293038 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6834ec7-952f-4c89-b194-aa6f61a69ff8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:52.322759 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1340/2681] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:11:52 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:11:52.323930 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f342c1f-0122-45b7-ac2f-7106c9d407ce admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:52.334275 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f342c1f-0122-45b7-ac2f-7106c9d407ce admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:52.362878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:11:52.397234 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3651b-5e38-4669-8d19-f707c3951b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:52.397931 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3651b-5e38-4669-8d19-f707c3951b7d admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:52.398217 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3651b-5e38-4669-8d19-f707c3951b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:52.398494 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3651b-5e38-4669-8d19-f707c3951b7d admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:52.399214 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d3651b-5e38-4669-8d19-f707c3951b7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:52.401131 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d3651b-5e38-4669-8d19-f707c3951b7d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:52.427944 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1342/2682] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:11:52 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:11:52.430071 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6834ec7-952f-4c89-b194-aa6f61a69ff8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:52.440870 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6834ec7-952f-4c89-b194-aa6f61a69ff8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:52.446032 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eac7b9-f0a0-4306-9113-a552b47e3f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:52.446759 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eac7b9-f0a0-4306-9113-a552b47e3f22 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:52.446991 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eac7b9-f0a0-4306-9113-a552b47e3f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:52.447285 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eac7b9-f0a0-4306-9113-a552b47e3f22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:52.448085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eac7b9-f0a0-4306-9113-a552b47e3f22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:52.449857 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eac7b9-f0a0-4306-9113-a552b47e3f22 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:52.461880 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1341/2683] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:11:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:11:52.466177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3651b-5e38-4669-8d19-f707c3951b7d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:52.477444 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3651b-5e38-4669-8d19-f707c3951b7d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:52.482303 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de081c54-011a-4176-bd01-409726ec4505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:52.482948 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de081c54-011a-4176-bd01-409726ec4505 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:52.483236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de081c54-011a-4176-bd01-409726ec4505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:52.483519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de081c54-011a-4176-bd01-409726ec4505 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:52.484287 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de081c54-011a-4176-bd01-409726ec4505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:52.485986 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de081c54-011a-4176-bd01-409726ec4505 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:52.497078 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1343/2684] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:11:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:11:52.501157 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eac7b9-f0a0-4306-9113-a552b47e3f22 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:52.511457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eac7b9-f0a0-4306-9113-a552b47e3f22 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:52.516282 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530e0353-2108-4e8a-b6c8-79e30960f05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:52.517188 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e0353-2108-4e8a-b6c8-79e30960f05c admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:52.517473 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e0353-2108-4e8a-b6c8-79e30960f05c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:52.517781 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e0353-2108-4e8a-b6c8-79e30960f05c admin admin] PATH_INFO: `/v3/projects/eeed79f9dc51474aa4dbfce9ee8c6acd/users/d1edd0e1953940a2badaf18e69074fe6/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:52.533369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530e0353-2108-4e8a-b6c8-79e30960f05c admin admin] RBAC: Authorizing `identity:create_grant(project_id=eeed79f9dc51474aa4dbfce9ee8c6acd, user_id=d1edd0e1953940a2badaf18e69074fe6, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:52.536170 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530e0353-2108-4e8a-b6c8-79e30960f05c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:52.549258 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1342/2685] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:11:52 2022] PUT /identity/v3/projects/eeed79f9dc51474aa4dbfce9ee8c6acd/users/d1edd0e1953940a2badaf18e69074fe6/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:11:52.562009 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79ff29-32b1-4beb-ae91-505884d5ec72 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:52.562307 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79ff29-32b1-4beb-ae91-505884d5ec72 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:52.562595 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79ff29-32b1-4beb-ae91-505884d5ec72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:52.589415 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4c79ff29-32b1-4beb-ae91-505884d5ec72 None None] MFA Rules not processed for user `d1edd0e1953940a2badaf18e69074fe6`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:11:52.590675 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4c79ff29-32b1-4beb-ae91-505884d5ec72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:52.659214 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1344/2686] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:11:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:11:52.671061 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530e0353-2108-4e8a-b6c8-79e30960f05c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:11:52.672724 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-530e0353-2108-4e8a-b6c8-79e30960f05c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:11:52.729254 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530e0353-2108-4e8a-b6c8-79e30960f05c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:11:52.732892 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7c60a-aaea-4f13-a710-626a4df84972 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73a7d5f37e144768d7cc53f13cd5157', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:11:52.733528 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7c60a-aaea-4f13-a710-626a4df84972 service glance] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:11:52.733838 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7c60a-aaea-4f13-a710-626a4df84972 service glance] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:11:52.734113 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7c60a-aaea-4f13-a710-626a4df84972 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:11:52.742822 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d7c60a-aaea-4f13-a710-626a4df84972 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:11:52.744252 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d7c60a-aaea-4f13-a710-626a4df84972 service glance] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:11:52.752915 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1343/2687] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:11:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:12:02.656304 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:12:02.749098 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=6,keystone:SELECT=67,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:12:16.561532 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79ff29-32b1-4beb-ae91-505884d5ec72 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:16.625231 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79ff29-32b1-4beb-ae91-505884d5ec72 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:16.629719 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b429a2-326a-4235-ad95-09a7f91323f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:16.630587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b429a2-326a-4235-ad95-09a7f91323f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:16.631424 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b429a2-326a-4235-ad95-09a7f91323f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:16.631701 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b429a2-326a-4235-ad95-09a7f91323f8 admin admin] PATH_INFO: `/v3/users/d1edd0e1953940a2badaf18e69074fe6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:16.632884 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b429a2-326a-4235-ad95-09a7f91323f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1edd0e1953940a2badaf18e69074fe6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:16.634827 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b429a2-326a-4235-ad95-09a7f91323f8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:16.679787 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b9b429a2-326a-4235-ad95-09a7f91323f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1edd0e1953940a2badaf18e69074fe6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:16.680446 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b9b429a2-326a-4235-ad95-09a7f91323f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1edd0e1953940a2badaf18e69074fe6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:16.681049 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b9b429a2-326a-4235-ad95-09a7f91323f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1edd0e1953940a2badaf18e69074fe6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:16.687943 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b9b429a2-326a-4235-ad95-09a7f91323f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1edd0e1953940a2badaf18e69074fe6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:16.695686 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b9b429a2-326a-4235-ad95-09a7f91323f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1edd0e1953940a2badaf18e69074fe6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:16.712634 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1345/2688] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:12:16 2022] DELETE /identity/v3/users/d1edd0e1953940a2badaf18e69074fe6 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:12:16.717085 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb1b28-5bbe-425d-9cf6-d55bfe9b5ab7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:16.717599 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb1b28-5bbe-425d-9cf6-d55bfe9b5ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:16.717713 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb1b28-5bbe-425d-9cf6-d55bfe9b5ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:16.735460 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-19bb1b28-5bbe-425d-9cf6-d55bfe9b5ab7 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:12:16.736661 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-19bb1b28-5bbe-425d-9cf6-d55bfe9b5ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:16.795020 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1344/2689] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:12:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:12:16.804626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b429a2-326a-4235-ad95-09a7f91323f8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:16.806067 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b9b429a2-326a-4235-ad95-09a7f91323f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:16.862980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b429a2-326a-4235-ad95-09a7f91323f8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:16.866685 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d7449-1661-4932-941c-6b465c5fb06a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:16.867334 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d7449-1661-4932-941c-6b465c5fb06a service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:16.867627 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d7449-1661-4932-941c-6b465c5fb06a service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:16.867912 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d7449-1661-4932-941c-6b465c5fb06a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:16.877288 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d7449-1661-4932-941c-6b465c5fb06a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:16.878767 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d7449-1661-4932-941c-6b465c5fb06a service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:16.898368 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1346/2690] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:12:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:12:17.175435 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb1b28-5bbe-425d-9cf6-d55bfe9b5ab7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:17.176928 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-19bb1b28-5bbe-425d-9cf6-d55bfe9b5ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:17.189707 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb1b28-5bbe-425d-9cf6-d55bfe9b5ab7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:17.195366 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a1e6f-e036-4fdc-9ecc-7bccda75faaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:17.196231 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a1e6f-e036-4fdc-9ecc-7bccda75faaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:17.196515 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a1e6f-e036-4fdc-9ecc-7bccda75faaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:17.196795 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a1e6f-e036-4fdc-9ecc-7bccda75faaa admin admin] PATH_INFO: `/v3/projects/eeed79f9dc51474aa4dbfce9ee8c6acd` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:17.198025 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a1e6f-e036-4fdc-9ecc-7bccda75faaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=eeed79f9dc51474aa4dbfce9ee8c6acd)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:17.199956 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a1e6f-e036-4fdc-9ecc-7bccda75faaa admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:17.259413 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3f6a1e6f-e036-4fdc-9ecc-7bccda75faaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eeed79f9dc51474aa4dbfce9ee8c6acd', 'request_id': 'req-3f6a1e6f-e036-4fdc-9ecc-7bccda75faaa'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:17.267311 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3f6a1e6f-e036-4fdc-9ecc-7bccda75faaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eeed79f9dc51474aa4dbfce9ee8c6acd', 'request_id': 'req-3f6a1e6f-e036-4fdc-9ecc-7bccda75faaa'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:17.273252 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3f6a1e6f-e036-4fdc-9ecc-7bccda75faaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eeed79f9dc51474aa4dbfce9ee8c6acd', 'request_id': 'req-3f6a1e6f-e036-4fdc-9ecc-7bccda75faaa'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:17.285180 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3f6a1e6f-e036-4fdc-9ecc-7bccda75faaa admin admin] The token cache is being invalidate because project eeed79f9dc51474aa4dbfce9ee8c6acd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:12:17.285733 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3f6a1e6f-e036-4fdc-9ecc-7bccda75faaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:17.286397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3f6a1e6f-e036-4fdc-9ecc-7bccda75faaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:17.288400 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1345/2691] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:12:17 2022] DELETE /identity/v3/projects/eeed79f9dc51474aa4dbfce9ee8c6acd => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:12:17.293703 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7d7a5-9a55-47e2-9941-405b4566a0a3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:17.294001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7d7a5-9a55-47e2-9941-405b4566a0a3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:17.294308 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7d7a5-9a55-47e2-9941-405b4566a0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:17.312181 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-80b7d7a5-9a55-47e2-9941-405b4566a0a3 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:12:17.313300 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-80b7d7a5-9a55-47e2-9941-405b4566a0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:17.331296 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:12:17.381795 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1347/2692] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:12:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:12:17.391734 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55f55d-80d6-423e-b58a-851c06510154 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:17.392148 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55f55d-80d6-423e-b58a-851c06510154 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:17.392494 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55f55d-80d6-423e-b58a-851c06510154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:17.412541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fa55f55d-80d6-423e-b58a-851c06510154 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:12:17.413861 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fa55f55d-80d6-423e-b58a-851c06510154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:17.422291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1346/2693] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:12:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:12:17.426517 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7d7a5-9a55-47e2-9941-405b4566a0a3 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:17.436677 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7d7a5-9a55-47e2-9941-405b4566a0a3 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:17.441438 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474d480-79a5-4972-a056-223ce18dcd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:17.442135 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474d480-79a5-4972-a056-223ce18dcd44 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:17.442413 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474d480-79a5-4972-a056-223ce18dcd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:17.442689 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474d480-79a5-4972-a056-223ce18dcd44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:17.443466 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b474d480-79a5-4972-a056-223ce18dcd44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:12:17.443776 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b474d480-79a5-4972-a056-223ce18dcd44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:17.445364 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b474d480-79a5-4972-a056-223ce18dcd44 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:17.458419 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1348/2694] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:12:17.463512 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa55f55d-80d6-423e-b58a-851c06510154 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:17.476398 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa55f55d-80d6-423e-b58a-851c06510154 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:17.481817 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696d235-aebb-46f4-b9fc-068852910634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:17.482472 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696d235-aebb-46f4-b9fc-068852910634 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:17.482755 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696d235-aebb-46f4-b9fc-068852910634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:17.483031 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696d235-aebb-46f4-b9fc-068852910634 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:17.483717 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e696d235-aebb-46f4-b9fc-068852910634 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:17.485653 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e696d235-aebb-46f4-b9fc-068852910634 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:17.516583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1347/2695] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:12:17 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:12:17.520495 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474d480-79a5-4972-a056-223ce18dcd44 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:17.533228 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474d480-79a5-4972-a056-223ce18dcd44 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:17.597464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964e99c-6606-4ad8-9b39-757dbb278f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:17.598534 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964e99c-6606-4ad8-9b39-757dbb278f47 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:17.598828 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964e99c-6606-4ad8-9b39-757dbb278f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:17.599116 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964e99c-6606-4ad8-9b39-757dbb278f47 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:17.600342 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6964e99c-6606-4ad8-9b39-757dbb278f47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:17.602306 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6964e99c-6606-4ad8-9b39-757dbb278f47 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:17.625245 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1349/2696] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:12:17 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:12:17.629809 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696d235-aebb-46f4-b9fc-068852910634 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:17.642281 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696d235-aebb-46f4-b9fc-068852910634 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:17.646941 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5bd52d-5a04-496f-bf76-53d53914c8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:17.647593 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5bd52d-5a04-496f-bf76-53d53914c8ae admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:17.647876 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5bd52d-5a04-496f-bf76-53d53914c8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:17.648150 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5bd52d-5a04-496f-bf76-53d53914c8ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:17.648870 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5bd52d-5a04-496f-bf76-53d53914c8ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:17.650498 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5bd52d-5a04-496f-bf76-53d53914c8ae admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:17.660963 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1348/2697] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:12:17.665752 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964e99c-6606-4ad8-9b39-757dbb278f47 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:17.675769 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964e99c-6606-4ad8-9b39-757dbb278f47 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:17.680935 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad5463-a590-49ce-8e64-b42a66c4f9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:17.681593 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad5463-a590-49ce-8e64-b42a66c4f9bc admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:17.681891 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad5463-a590-49ce-8e64-b42a66c4f9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:17.682169 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad5463-a590-49ce-8e64-b42a66c4f9bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:17.683010 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad5463-a590-49ce-8e64-b42a66c4f9bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:17.684603 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad5463-a590-49ce-8e64-b42a66c4f9bc admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:17.694337 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1350/2698] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 16:12:17.698325 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5bd52d-5a04-496f-bf76-53d53914c8ae admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:17.708642 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5bd52d-5a04-496f-bf76-53d53914c8ae admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:17.713269 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443c0df-e2ed-47f6-9c55-100325196a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:17.714174 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443c0df-e2ed-47f6-9c55-100325196a9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:17.714460 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443c0df-e2ed-47f6-9c55-100325196a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:17.714736 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443c0df-e2ed-47f6-9c55-100325196a9e admin admin] PATH_INFO: `/v3/projects/5ad1ad798ef54f788b5eb3ad047c1836/users/46a6b952398a48db9314d3bf597e9eee/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:17.729985 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a443c0df-e2ed-47f6-9c55-100325196a9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ad1ad798ef54f788b5eb3ad047c1836, user_id=46a6b952398a48db9314d3bf597e9eee, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:17.732659 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a443c0df-e2ed-47f6-9c55-100325196a9e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:17.746392 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1349/2699] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:12:17 2022] PUT /identity/v3/projects/5ad1ad798ef54f788b5eb3ad047c1836/users/46a6b952398a48db9314d3bf597e9eee/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:12:17.811271 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6149582-457c-4847-82b1-667b65f28e23 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:17.811567 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6149582-457c-4847-82b1-667b65f28e23 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:17.811852 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6149582-457c-4847-82b1-667b65f28e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:17.838403 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a6149582-457c-4847-82b1-667b65f28e23 None None] MFA Rules not processed for user `46a6b952398a48db9314d3bf597e9eee`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:12:17.839561 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a6149582-457c-4847-82b1-667b65f28e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:17.905406 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1351/2700] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:12:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:12:17.916947 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443c0df-e2ed-47f6-9c55-100325196a9e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:17.918736 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a443c0df-e2ed-47f6-9c55-100325196a9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:17.944602 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:12:17.984452 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443c0df-e2ed-47f6-9c55-100325196a9e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:17.988211 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36657a82-8913-45b3-95b9-126c2e161bf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:17.988875 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36657a82-8913-45b3-95b9-126c2e161bf9 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:17.989175 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36657a82-8913-45b3-95b9-126c2e161bf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:17.989480 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36657a82-8913-45b3-95b9-126c2e161bf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:17.997908 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36657a82-8913-45b3-95b9-126c2e161bf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:17.999369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36657a82-8913-45b3-95b9-126c2e161bf9 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:18.008448 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1350/2701] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:12:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:12:19.359531 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74009d-c9f1-45bb-a09c-d70c651f8a08 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:19.360044 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74009d-c9f1-45bb-a09c-d70c651f8a08 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:19.360143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74009d-c9f1-45bb-a09c-d70c651f8a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:19.377715 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0a74009d-c9f1-45bb-a09c-d70c651f8a08 None None] MFA Rules not processed for user `46a6b952398a48db9314d3bf597e9eee`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:12:19.378903 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0a74009d-c9f1-45bb-a09c-d70c651f8a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:19.385658 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1352/2702] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:12:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:12:22.112536 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36657a82-8913-45b3-95b9-126c2e161bf9 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:22.121284 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-36657a82-8913-45b3-95b9-126c2e161bf9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:22.185820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36657a82-8913-45b3-95b9-126c2e161bf9 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:22.190382 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf8b985-58e9-4777-96c5-06bdc790f0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:22.191291 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8b985-58e9-4777-96c5-06bdc790f0f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:22.191577 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8b985-58e9-4777-96c5-06bdc790f0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:22.191858 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8b985-58e9-4777-96c5-06bdc790f0f3 admin admin] PATH_INFO: `/v3/users/12660992e7ec4912b77a82c4f3edc607` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:22.193650 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf8b985-58e9-4777-96c5-06bdc790f0f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12660992e7ec4912b77a82c4f3edc607)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:22.195668 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf8b985-58e9-4777-96c5-06bdc790f0f3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:22.258429 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8cf8b985-58e9-4777-96c5-06bdc790f0f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12660992e7ec4912b77a82c4f3edc607'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:22.259379 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8cf8b985-58e9-4777-96c5-06bdc790f0f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12660992e7ec4912b77a82c4f3edc607'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:22.262125 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8cf8b985-58e9-4777-96c5-06bdc790f0f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12660992e7ec4912b77a82c4f3edc607'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:22.274878 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8cf8b985-58e9-4777-96c5-06bdc790f0f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12660992e7ec4912b77a82c4f3edc607'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:22.285908 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8cf8b985-58e9-4777-96c5-06bdc790f0f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12660992e7ec4912b77a82c4f3edc607'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:22.312802 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1351/2703] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:12:22 2022] DELETE /identity/v3/users/12660992e7ec4912b77a82c4f3edc607 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:12:22.314243 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4babd42-09c7-4f47-9830-b00d1221a60d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:22.314705 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4babd42-09c7-4f47-9830-b00d1221a60d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:22.314917 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4babd42-09c7-4f47-9830-b00d1221a60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:22.351780 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f4babd42-09c7-4f47-9830-b00d1221a60d None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:12:22.353169 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f4babd42-09c7-4f47-9830-b00d1221a60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:22.360630 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:12:22.431845 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1353/2704] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:12:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:12:22.438368 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf8b985-58e9-4777-96c5-06bdc790f0f3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:22.439811 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8cf8b985-58e9-4777-96c5-06bdc790f0f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:22.492166 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:12:22.505694 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf8b985-58e9-4777-96c5-06bdc790f0f3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:22.510082 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccfee97-800f-4528-af55-4bc29da45dd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:22.510759 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfee97-800f-4528-af55-4bc29da45dd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:22.511069 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfee97-800f-4528-af55-4bc29da45dd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:22.511353 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfee97-800f-4528-af55-4bc29da45dd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:22.521738 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccfee97-800f-4528-af55-4bc29da45dd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:22.523199 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccfee97-800f-4528-af55-4bc29da45dd9 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:22.532907 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1352/2705] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:12:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:12:22.820307 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4babd42-09c7-4f47-9830-b00d1221a60d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:22.825403 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f4babd42-09c7-4f47-9830-b00d1221a60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:22.834956 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4babd42-09c7-4f47-9830-b00d1221a60d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:22.839586 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7023e1a4-160d-4d2b-a258-9f6988d4eb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:22.840514 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023e1a4-160d-4d2b-a258-9f6988d4eb7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:22.840802 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023e1a4-160d-4d2b-a258-9f6988d4eb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:22.841081 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023e1a4-160d-4d2b-a258-9f6988d4eb7f admin admin] PATH_INFO: `/v3/projects/3227949ec1064cd0bff5468018ed6bc9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:22.842299 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7023e1a4-160d-4d2b-a258-9f6988d4eb7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3227949ec1064cd0bff5468018ed6bc9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:22.844213 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7023e1a4-160d-4d2b-a258-9f6988d4eb7f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:22.892088 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7023e1a4-160d-4d2b-a258-9f6988d4eb7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3227949ec1064cd0bff5468018ed6bc9', 'request_id': 'req-7023e1a4-160d-4d2b-a258-9f6988d4eb7f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:22.898741 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7023e1a4-160d-4d2b-a258-9f6988d4eb7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3227949ec1064cd0bff5468018ed6bc9', 'request_id': 'req-7023e1a4-160d-4d2b-a258-9f6988d4eb7f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:22.905242 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7023e1a4-160d-4d2b-a258-9f6988d4eb7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3227949ec1064cd0bff5468018ed6bc9', 'request_id': 'req-7023e1a4-160d-4d2b-a258-9f6988d4eb7f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:22.915411 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7023e1a4-160d-4d2b-a258-9f6988d4eb7f admin admin] The token cache is being invalidate because project 3227949ec1064cd0bff5468018ed6bc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:12:22.915698 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7023e1a4-160d-4d2b-a258-9f6988d4eb7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:22.916316 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7023e1a4-160d-4d2b-a258-9f6988d4eb7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:22.917939 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1354/2706] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:12:22 2022] DELETE /identity/v3/projects/3227949ec1064cd0bff5468018ed6bc9 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:12:22.922701 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ade21-bae4-4c57-a3e9-b1a9d9ecfa27 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:22.923058 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ade21-bae4-4c57-a3e9-b1a9d9ecfa27 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:22.923403 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ade21-bae4-4c57-a3e9-b1a9d9ecfa27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:22.940619 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-bd9ade21-bae4-4c57-a3e9-b1a9d9ecfa27 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:12:22.941939 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bd9ade21-bae4-4c57-a3e9-b1a9d9ecfa27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:23.001279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1353/2707] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:12:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:12:23.009682 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e95b6-c2c1-4368-8387-3c63fb73e757 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:23.009973 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e95b6-c2c1-4368-8387-3c63fb73e757 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:23.010259 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e95b6-c2c1-4368-8387-3c63fb73e757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:23.029691 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d89e95b6-c2c1-4368-8387-3c63fb73e757 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:12:23.030888 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d89e95b6-c2c1-4368-8387-3c63fb73e757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:23.038884 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1355/2708] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:12:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:12:23.042773 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ade21-bae4-4c57-a3e9-b1a9d9ecfa27 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:23.052333 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ade21-bae4-4c57-a3e9-b1a9d9ecfa27 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:23.056917 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc5b84-976d-4ec6-89f8-3d4a5eb1c897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:23.057614 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc5b84-976d-4ec6-89f8-3d4a5eb1c897 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:23.057909 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc5b84-976d-4ec6-89f8-3d4a5eb1c897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:23.058187 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc5b84-976d-4ec6-89f8-3d4a5eb1c897 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:23.058917 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc5b84-976d-4ec6-89f8-3d4a5eb1c897 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:12:23.059222 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc5b84-976d-4ec6-89f8-3d4a5eb1c897 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:23.060834 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc5b84-976d-4ec6-89f8-3d4a5eb1c897 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:23.072679 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1354/2709] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:12:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:12:23.076709 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e95b6-c2c1-4368-8387-3c63fb73e757 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:23.086398 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e95b6-c2c1-4368-8387-3c63fb73e757 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:23.090954 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca79d1-a67e-4819-b34b-702294290f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:23.091592 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca79d1-a67e-4819-b34b-702294290f25 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:23.091873 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca79d1-a67e-4819-b34b-702294290f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:23.092146 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca79d1-a67e-4819-b34b-702294290f25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:23.092856 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bca79d1-a67e-4819-b34b-702294290f25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:23.094778 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bca79d1-a67e-4819-b34b-702294290f25 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:23.120651 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1356/2710] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:12:23 2022] POST /identity/v3/projects => generated 387 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:12:23.124548 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc5b84-976d-4ec6-89f8-3d4a5eb1c897 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:23.136540 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc5b84-976d-4ec6-89f8-3d4a5eb1c897 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:23.191681 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33cdda7-b037-45ee-be58-26da4022bb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:23.192278 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33cdda7-b037-45ee-be58-26da4022bb6c admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:23.192561 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33cdda7-b037-45ee-be58-26da4022bb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:23.193447 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33cdda7-b037-45ee-be58-26da4022bb6c admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:23.194176 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33cdda7-b037-45ee-be58-26da4022bb6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:23.196091 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33cdda7-b037-45ee-be58-26da4022bb6c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:23.214980 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1355/2711] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:12:23 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:12:23.219164 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca79d1-a67e-4819-b34b-702294290f25 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:23.230404 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca79d1-a67e-4819-b34b-702294290f25 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:23.235407 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774add9b-cb78-4525-83f6-5160475dd9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:23.236084 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774add9b-cb78-4525-83f6-5160475dd9b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:23.236376 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774add9b-cb78-4525-83f6-5160475dd9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:23.236665 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774add9b-cb78-4525-83f6-5160475dd9b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:23.237507 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774add9b-cb78-4525-83f6-5160475dd9b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:23.239149 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774add9b-cb78-4525-83f6-5160475dd9b3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:23.250398 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1357/2712] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:12:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:12:23.254499 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33cdda7-b037-45ee-be58-26da4022bb6c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:23.265056 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33cdda7-b037-45ee-be58-26da4022bb6c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:23.269696 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e196e2-b5b5-4d52-ade8-755bdcb128e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:23.270343 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e196e2-b5b5-4d52-ade8-755bdcb128e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:23.270625 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e196e2-b5b5-4d52-ade8-755bdcb128e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:23.270897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e196e2-b5b5-4d52-ade8-755bdcb128e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:23.271615 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e196e2-b5b5-4d52-ade8-755bdcb128e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:23.273188 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e196e2-b5b5-4d52-ade8-755bdcb128e4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:23.282884 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1356/2713] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:12:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 16:12:23.286924 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774add9b-cb78-4525-83f6-5160475dd9b3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:23.296727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774add9b-cb78-4525-83f6-5160475dd9b3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:23.301258 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a5bc9f-64a4-4563-95f8-290bfcfa190e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:23.302657 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5bc9f-64a4-4563-95f8-290bfcfa190e admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:23.302938 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5bc9f-64a4-4563-95f8-290bfcfa190e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:23.303210 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5bc9f-64a4-4563-95f8-290bfcfa190e admin admin] PATH_INFO: `/v3/projects/cb7263dfdcdd4253bb3827262a72600b/users/d572ae6eb90f403ebd98e2bdb9d5da35/roles/54195e660fc349978b39d5794765a095` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:23.317396 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a5bc9f-64a4-4563-95f8-290bfcfa190e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb7263dfdcdd4253bb3827262a72600b, user_id=d572ae6eb90f403ebd98e2bdb9d5da35, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:23.320672 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a5bc9f-64a4-4563-95f8-290bfcfa190e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:23.333817 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1358/2714] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:12:23 2022] PUT /identity/v3/projects/cb7263dfdcdd4253bb3827262a72600b/users/d572ae6eb90f403ebd98e2bdb9d5da35/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:12:23.400843 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90245734-57af-43e6-8831-768a4bbac294 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:23.401136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90245734-57af-43e6-8831-768a4bbac294 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:23.402033 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90245734-57af-43e6-8831-768a4bbac294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:23.434306 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-90245734-57af-43e6-8831-768a4bbac294 None None] MFA Rules not processed for user `d572ae6eb90f403ebd98e2bdb9d5da35`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:12:23.435695 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-90245734-57af-43e6-8831-768a4bbac294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:23.481186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:12:23.514597 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1357/2715] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:12:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:12:23.526693 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a5bc9f-64a4-4563-95f8-290bfcfa190e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:23.528551 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-25a5bc9f-64a4-4563-95f8-290bfcfa190e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:23.568968 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:12:23.597566 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a5bc9f-64a4-4563-95f8-290bfcfa190e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:23.603000 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02dae07-72fd-4fe9-82cd-570a6ced801e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:23.604075 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02dae07-72fd-4fe9-82cd-570a6ced801e service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:23.604432 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02dae07-72fd-4fe9-82cd-570a6ced801e service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:23.605240 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02dae07-72fd-4fe9-82cd-570a6ced801e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:23.614134 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02dae07-72fd-4fe9-82cd-570a6ced801e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:23.615763 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02dae07-72fd-4fe9-82cd-570a6ced801e service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:23.626422 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1359/2716] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:12:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:12:31.602191 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90245734-57af-43e6-8831-768a4bbac294 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:31.603622 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-90245734-57af-43e6-8831-768a4bbac294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:31.646342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90245734-57af-43e6-8831-768a4bbac294 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:31.650279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5effc-b3a3-489c-bf40-5c11a15d24af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:31.650931 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5effc-b3a3-489c-bf40-5c11a15d24af service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:31.651215 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5effc-b3a3-489c-bf40-5c11a15d24af service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:31.651493 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5effc-b3a3-489c-bf40-5c11a15d24af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:31.653274 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d3c5effc-b3a3-489c-bf40-5c11a15d24af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:31.726281 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c5effc-b3a3-489c-bf40-5c11a15d24af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:31.727732 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c5effc-b3a3-489c-bf40-5c11a15d24af service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:31.736761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1358/2717] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:12:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:12:33.621853 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=12 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:12:39.505040 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02dae07-72fd-4fe9-82cd-570a6ced801e service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:39.507221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a02dae07-72fd-4fe9-82cd-570a6ced801e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:39.663848 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02dae07-72fd-4fe9-82cd-570a6ced801e service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:39.668586 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2eb0e-6ba0-4129-bb5c-34cfd990dc75 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822d9d3521db4a9688f51c6313c7dd74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:39.669196 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2eb0e-6ba0-4129-bb5c-34cfd990dc75 service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:39.669484 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2eb0e-6ba0-4129-bb5c-34cfd990dc75 service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:39.669903 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2eb0e-6ba0-4129-bb5c-34cfd990dc75 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:39.671762 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2de2eb0e-6ba0-4129-bb5c-34cfd990dc75 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:39.684170 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de2eb0e-6ba0-4129-bb5c-34cfd990dc75 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:39.685776 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de2eb0e-6ba0-4129-bb5c-34cfd990dc75 service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:39.695966 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1360/2718] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:12:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:12:41.367167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e47629-6ae2-49a6-b0fb-6efc552dd4b8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:41.367618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e47629-6ae2-49a6-b0fb-6efc552dd4b8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:41.367714 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e47629-6ae2-49a6-b0fb-6efc552dd4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:41.395282 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-83e47629-6ae2-49a6-b0fb-6efc552dd4b8 None None] MFA Rules not processed for user `6edbad98ed0f4823b9955f9f0e270149`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:12:41.396503 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-83e47629-6ae2-49a6-b0fb-6efc552dd4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:41.458523 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1359/2719] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 21 16:12:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:12:41.503999 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2eb0e-6ba0-4129-bb5c-34cfd990dc75 service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:41.506137 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2de2eb0e-6ba0-4129-bb5c-34cfd990dc75 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:41.573557 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2eb0e-6ba0-4129-bb5c-34cfd990dc75 service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:41.578523 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b98eee-b6fc-491e-86ee-7f28e0d5bf22 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5bfae2cb30843b1b1a0343efef38638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:41.579165 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b98eee-b6fc-491e-86ee-7f28e0d5bf22 service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:41.579448 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b98eee-b6fc-491e-86ee-7f28e0d5bf22 service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:41.579724 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b98eee-b6fc-491e-86ee-7f28e0d5bf22 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:41.587687 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b98eee-b6fc-491e-86ee-7f28e0d5bf22 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:41.589253 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b98eee-b6fc-491e-86ee-7f28e0d5bf22 service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:41.597395 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1361/2720] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 16:12:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 16:12:41.731773 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b61eb-a918-42e5-8fbd-7197678f4838 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:41.732063 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b61eb-a918-42e5-8fbd-7197678f4838 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:41.732342 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b61eb-a918-42e5-8fbd-7197678f4838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:41.754082 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e20b61eb-a918-42e5-8fbd-7197678f4838 None None] MFA Rules not processed for user `762f5eba8cbe4bb9b40062e6cd1e9929`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:12:41.755234 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e20b61eb-a918-42e5-8fbd-7197678f4838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:41.765419 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1360/2721] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:12:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:12:42.133710 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153fb8b-2e08-4f7e-af05-2aa4cbb2e9ff None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:42.134005 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153fb8b-2e08-4f7e-af05-2aa4cbb2e9ff None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:42.134281 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153fb8b-2e08-4f7e-af05-2aa4cbb2e9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:42.151848 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4153fb8b-2e08-4f7e-af05-2aa4cbb2e9ff None None] MFA Rules not processed for user `6edbad98ed0f4823b9955f9f0e270149`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:12:42.153047 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4153fb8b-2e08-4f7e-af05-2aa4cbb2e9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:42.160690 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1362/2722] 10.222.0.52 () {58 vars in 967 bytes} [Thu Jul 21 16:12:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:12:42.174203 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b61eb-a918-42e5-8fbd-7197678f4838 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:42.186969 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b61eb-a918-42e5-8fbd-7197678f4838 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:42.192836 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc4d55d-bed9-498f-8021-7705295cf5db service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5bfae2cb30843b1b1a0343efef38638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:42.193482 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc4d55d-bed9-498f-8021-7705295cf5db service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:42.193792 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc4d55d-bed9-498f-8021-7705295cf5db service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:42.194071 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc4d55d-bed9-498f-8021-7705295cf5db service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:42.202659 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc4d55d-bed9-498f-8021-7705295cf5db service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:42.204328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc4d55d-bed9-498f-8021-7705295cf5db service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:42.213196 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1361/2723] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 16:12:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 16:12:42.420568 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ee028-2a71-47d6-b167-2aa0a2e9adfd None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:42.420892 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ee028-2a71-47d6-b167-2aa0a2e9adfd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:42.421252 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ee028-2a71-47d6-b167-2aa0a2e9adfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:42.440291 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8f9ee028-2a71-47d6-b167-2aa0a2e9adfd None None] MFA Rules not processed for user `6edbad98ed0f4823b9955f9f0e270149`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:12:42.441608 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8f9ee028-2a71-47d6-b167-2aa0a2e9adfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:42.449659 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1363/2724] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 21 16:12:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:12:42.463028 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc4d55d-bed9-498f-8021-7705295cf5db service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:42.474003 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc4d55d-bed9-498f-8021-7705295cf5db service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:42.479054 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf320d8f-3fec-4751-a7aa-f85d9a19d587 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5bfae2cb30843b1b1a0343efef38638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:42.479688 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf320d8f-3fec-4751-a7aa-f85d9a19d587 service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:42.479997 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf320d8f-3fec-4751-a7aa-f85d9a19d587 service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:42.480293 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf320d8f-3fec-4751-a7aa-f85d9a19d587 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:42.485739 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:12:42.491079 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf320d8f-3fec-4751-a7aa-f85d9a19d587 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:42.492684 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf320d8f-3fec-4751-a7aa-f85d9a19d587 service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:42.501131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1362/2725] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 16:12:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 16:12:42.829002 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97137a7b-2d7f-40ef-996e-c265aea08efb None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:42.829455 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97137a7b-2d7f-40ef-996e-c265aea08efb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:42.829567 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97137a7b-2d7f-40ef-996e-c265aea08efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:42.848221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-97137a7b-2d7f-40ef-996e-c265aea08efb None None] MFA Rules not processed for user `6edbad98ed0f4823b9955f9f0e270149`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:12:42.850156 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-97137a7b-2d7f-40ef-996e-c265aea08efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:42.857636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1364/2726] 10.222.0.52 () {58 vars in 967 bytes} [Thu Jul 21 16:12:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:12:42.869964 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf320d8f-3fec-4751-a7aa-f85d9a19d587 service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:42.880657 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf320d8f-3fec-4751-a7aa-f85d9a19d587 service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:42.888158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae889c-b180-4ef1-922f-fc241df46ef8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5bfae2cb30843b1b1a0343efef38638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:42.888809 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae889c-b180-4ef1-922f-fc241df46ef8 service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:42.889099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae889c-b180-4ef1-922f-fc241df46ef8 service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:42.889497 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae889c-b180-4ef1-922f-fc241df46ef8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:42.899050 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae889c-b180-4ef1-922f-fc241df46ef8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:42.900756 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae889c-b180-4ef1-922f-fc241df46ef8 service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:42.909454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1363/2727] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 16:12:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 16:12:44.023771 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97137a7b-2d7f-40ef-996e-c265aea08efb None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:44.034812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:12:44.115137 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97137a7b-2d7f-40ef-996e-c265aea08efb None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:44.120850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba688707-6366-4f42-8c5e-1ef6e8f6b20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:44.121895 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba688707-6366-4f42-8c5e-1ef6e8f6b20f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:44.122185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba688707-6366-4f42-8c5e-1ef6e8f6b20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:44.122810 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba688707-6366-4f42-8c5e-1ef6e8f6b20f admin admin] PATH_INFO: `/v3/users/d572ae6eb90f403ebd98e2bdb9d5da35` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:44.124212 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba688707-6366-4f42-8c5e-1ef6e8f6b20f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d572ae6eb90f403ebd98e2bdb9d5da35)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:44.126294 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba688707-6366-4f42-8c5e-1ef6e8f6b20f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:44.177036 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ba688707-6366-4f42-8c5e-1ef6e8f6b20f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd572ae6eb90f403ebd98e2bdb9d5da35'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:44.177736 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ba688707-6366-4f42-8c5e-1ef6e8f6b20f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd572ae6eb90f403ebd98e2bdb9d5da35'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:44.178382 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ba688707-6366-4f42-8c5e-1ef6e8f6b20f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd572ae6eb90f403ebd98e2bdb9d5da35'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:44.185431 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ba688707-6366-4f42-8c5e-1ef6e8f6b20f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd572ae6eb90f403ebd98e2bdb9d5da35'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:44.193405 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ba688707-6366-4f42-8c5e-1ef6e8f6b20f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd572ae6eb90f403ebd98e2bdb9d5da35'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:44.211687 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1365/2728] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:12:44 2022] DELETE /identity/v3/users/d572ae6eb90f403ebd98e2bdb9d5da35 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:12:44.216230 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dba2a4-c561-4bfd-8c93-222e5fdaa073 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:44.216784 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dba2a4-c561-4bfd-8c93-222e5fdaa073 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:44.216895 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dba2a4-c561-4bfd-8c93-222e5fdaa073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:44.258726 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-38dba2a4-c561-4bfd-8c93-222e5fdaa073 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:12:44.259953 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-38dba2a4-c561-4bfd-8c93-222e5fdaa073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:44.318817 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1364/2729] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:12:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:12:44.328288 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba688707-6366-4f42-8c5e-1ef6e8f6b20f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:44.329718 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ba688707-6366-4f42-8c5e-1ef6e8f6b20f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:44.408579 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba688707-6366-4f42-8c5e-1ef6e8f6b20f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:44.412262 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3713fe-218e-4ca5-85a8-f44c2c678d05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:44.412907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3713fe-218e-4ca5-85a8-f44c2c678d05 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:44.413196 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3713fe-218e-4ca5-85a8-f44c2c678d05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:44.413472 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3713fe-218e-4ca5-85a8-f44c2c678d05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:44.419363 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:12:44.423646 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3713fe-218e-4ca5-85a8-f44c2c678d05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:44.425067 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3713fe-218e-4ca5-85a8-f44c2c678d05 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:44.436508 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1366/2730] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:12:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:12:44.719186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dba2a4-c561-4bfd-8c93-222e5fdaa073 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:44.721157 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-38dba2a4-c561-4bfd-8c93-222e5fdaa073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:44.750465 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dba2a4-c561-4bfd-8c93-222e5fdaa073 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:44.758533 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b14ff-1fa5-45db-8ad9-be5128aa0832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:44.762613 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b14ff-1fa5-45db-8ad9-be5128aa0832 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:44.763023 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b14ff-1fa5-45db-8ad9-be5128aa0832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:44.763177 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b14ff-1fa5-45db-8ad9-be5128aa0832 admin admin] PATH_INFO: `/v3/projects/cb7263dfdcdd4253bb3827262a72600b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:44.765107 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b14ff-1fa5-45db-8ad9-be5128aa0832 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb7263dfdcdd4253bb3827262a72600b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:44.767833 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b14ff-1fa5-45db-8ad9-be5128aa0832 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:44.820502 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-078b14ff-1fa5-45db-8ad9-be5128aa0832 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb7263dfdcdd4253bb3827262a72600b', 'request_id': 'req-078b14ff-1fa5-45db-8ad9-be5128aa0832'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:44.828647 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-078b14ff-1fa5-45db-8ad9-be5128aa0832 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb7263dfdcdd4253bb3827262a72600b', 'request_id': 'req-078b14ff-1fa5-45db-8ad9-be5128aa0832'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:44.834987 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-078b14ff-1fa5-45db-8ad9-be5128aa0832 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb7263dfdcdd4253bb3827262a72600b', 'request_id': 'req-078b14ff-1fa5-45db-8ad9-be5128aa0832'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:44.845877 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-078b14ff-1fa5-45db-8ad9-be5128aa0832 admin admin] The token cache is being invalidate because project cb7263dfdcdd4253bb3827262a72600b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:12:44.846157 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-078b14ff-1fa5-45db-8ad9-be5128aa0832 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:44.847038 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-078b14ff-1fa5-45db-8ad9-be5128aa0832 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:44.848587 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1365/2731] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:12:44 2022] DELETE /identity/v3/projects/cb7263dfdcdd4253bb3827262a72600b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:12:44.853529 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfddcfb8-24d1-4b22-a6ab-8d83f55bd3f1 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:44.853858 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfddcfb8-24d1-4b22-a6ab-8d83f55bd3f1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:44.854144 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfddcfb8-24d1-4b22-a6ab-8d83f55bd3f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:44.906470 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-bfddcfb8-24d1-4b22-a6ab-8d83f55bd3f1 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:12:44.907707 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bfddcfb8-24d1-4b22-a6ab-8d83f55bd3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:44.968973 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1367/2732] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:12:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:12:44.975889 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b14ff-1fa5-45db-8ad9-be5128aa0832 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:44.976793 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153929b-9e79-4753-8b27-1e7e3a7a00a3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:44.977091 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153929b-9e79-4753-8b27-1e7e3a7a00a3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:44.977300 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-078b14ff-1fa5-45db-8ad9-be5128aa0832 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:44.977430 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153929b-9e79-4753-8b27-1e7e3a7a00a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:44.990873 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b14ff-1fa5-45db-8ad9-be5128aa0832 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:44.995655 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f01d1a-5c15-4365-8efa-c8b74a5e6500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:44.995822 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1153929b-9e79-4753-8b27-1e7e3a7a00a3 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:12:44.996529 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f01d1a-5c15-4365-8efa-c8b74a5e6500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:44.996815 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f01d1a-5c15-4365-8efa-c8b74a5e6500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:44.997008 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1153929b-9e79-4753-8b27-1e7e3a7a00a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:44.997110 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f01d1a-5c15-4365-8efa-c8b74a5e6500 admin admin] PATH_INFO: `/v3/users/46a6b952398a48db9314d3bf597e9eee` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:44.998466 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f01d1a-5c15-4365-8efa-c8b74a5e6500 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46a6b952398a48db9314d3bf597e9eee)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:45.000399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f01d1a-5c15-4365-8efa-c8b74a5e6500 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:45.005555 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1368/2733] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:12:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:12:45.009700 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153929b-9e79-4753-8b27-1e7e3a7a00a3 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:45.018697 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153929b-9e79-4753-8b27-1e7e3a7a00a3 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:45.023427 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4606ec-16ec-4cdf-99e2-fb1200247a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:45.024075 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4606ec-16ec-4cdf-99e2-fb1200247a52 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:45.024649 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4606ec-16ec-4cdf-99e2-fb1200247a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:45.024649 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4606ec-16ec-4cdf-99e2-fb1200247a52 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:45.025452 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4606ec-16ec-4cdf-99e2-fb1200247a52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:12:45.026160 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4606ec-16ec-4cdf-99e2-fb1200247a52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:45.028230 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4606ec-16ec-4cdf-99e2-fb1200247a52 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:45.042153 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1369/2734] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:12:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 16:12:45.046039 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4606ec-16ec-4cdf-99e2-fb1200247a52 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:45.046267 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:12:45.053685 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a7f01d1a-5c15-4365-8efa-c8b74a5e6500 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46a6b952398a48db9314d3bf597e9eee'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:45.055004 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a7f01d1a-5c15-4365-8efa-c8b74a5e6500 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46a6b952398a48db9314d3bf597e9eee'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:45.055731 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a7f01d1a-5c15-4365-8efa-c8b74a5e6500 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46a6b952398a48db9314d3bf597e9eee'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:45.057429 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4606ec-16ec-4cdf-99e2-fb1200247a52 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:45.062134 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c16e28f-dd64-483a-b042-9ae0bd641b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:45.062772 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16e28f-dd64-483a-b042-9ae0bd641b7f admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:45.063061 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16e28f-dd64-483a-b042-9ae0bd641b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:45.063344 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16e28f-dd64-483a-b042-9ae0bd641b7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:45.063803 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a7f01d1a-5c15-4365-8efa-c8b74a5e6500 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46a6b952398a48db9314d3bf597e9eee'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:45.064035 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c16e28f-dd64-483a-b042-9ae0bd641b7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:45.065964 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c16e28f-dd64-483a-b042-9ae0bd641b7f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:45.071288 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a7f01d1a-5c15-4365-8efa-c8b74a5e6500 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46a6b952398a48db9314d3bf597e9eee'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:45.090158 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1366/2735] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:12:44 2022] DELETE /identity/v3/users/46a6b952398a48db9314d3bf597e9eee => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:12:45.095539 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1370/2736] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:12:45 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:12:45.096989 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c16e28f-dd64-483a-b042-9ae0bd641b7f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:45.097359 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed86dd-3ca4-46df-ac9f-997d7c0374eb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:45.097672 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed86dd-3ca4-46df-ac9f-997d7c0374eb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:45.097965 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed86dd-3ca4-46df-ac9f-997d7c0374eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:45.098707 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6c16e28f-dd64-483a-b042-9ae0bd641b7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:45.115803 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-94ed86dd-3ca4-46df-ac9f-997d7c0374eb None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:12:45.116994 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-94ed86dd-3ca4-46df-ac9f-997d7c0374eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:45.163518 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c16e28f-dd64-483a-b042-9ae0bd641b7f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:45.168428 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695dbcf7-a553-46d2-a777-05d376239290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:45.169094 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695dbcf7-a553-46d2-a777-05d376239290 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:45.169381 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695dbcf7-a553-46d2-a777-05d376239290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:45.169703 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695dbcf7-a553-46d2-a777-05d376239290 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:45.170393 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695dbcf7-a553-46d2-a777-05d376239290 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:45.172322 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695dbcf7-a553-46d2-a777-05d376239290 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:45.177531 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1367/2737] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:12:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:12:45.188658 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed86dd-3ca4-46df-ac9f-997d7c0374eb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:45.190079 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-94ed86dd-3ca4-46df-ac9f-997d7c0374eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:45.195507 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1371/2738] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:12:45 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:12:45.199545 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695dbcf7-a553-46d2-a777-05d376239290 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:45.209466 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695dbcf7-a553-46d2-a777-05d376239290 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:45.214132 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc722e-5220-4a14-9637-d31317c7de80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:45.214775 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc722e-5220-4a14-9637-d31317c7de80 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:45.215067 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc722e-5220-4a14-9637-d31317c7de80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:45.215361 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc722e-5220-4a14-9637-d31317c7de80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:45.216063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc722e-5220-4a14-9637-d31317c7de80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:45.217669 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc722e-5220-4a14-9637-d31317c7de80 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:45.228606 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1372/2739] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 16:12:45.232837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc722e-5220-4a14-9637-d31317c7de80 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:45.243001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc722e-5220-4a14-9637-d31317c7de80 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:45.247978 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8de08-7130-4ca0-adfb-6fa60a071138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:45.248683 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8de08-7130-4ca0-adfb-6fa60a071138 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:45.248974 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8de08-7130-4ca0-adfb-6fa60a071138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:45.249252 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8de08-7130-4ca0-adfb-6fa60a071138 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:45.250007 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c8de08-7130-4ca0-adfb-6fa60a071138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:45.251608 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c8de08-7130-4ca0-adfb-6fa60a071138 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:45.253129 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:12:45.261232 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed86dd-3ca4-46df-ac9f-997d7c0374eb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:45.268178 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1373/2740] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 16:12:45.268349 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7195719-5bf1-4bef-a34b-ccd2e9e3db4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:45.268349 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7195719-5bf1-4bef-a34b-ccd2e9e3db4d service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:45.268349 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7195719-5bf1-4bef-a34b-ccd2e9e3db4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:45.268349 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7195719-5bf1-4bef-a34b-ccd2e9e3db4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:45.269834 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8de08-7130-4ca0-adfb-6fa60a071138 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:45.276949 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7195719-5bf1-4bef-a34b-ccd2e9e3db4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:45.278434 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7195719-5bf1-4bef-a34b-ccd2e9e3db4d service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:45.280083 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8de08-7130-4ca0-adfb-6fa60a071138 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:45.284702 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68965d-d8c2-44d4-b241-70c6a6f090e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:45.285606 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68965d-d8c2-44d4-b241-70c6a6f090e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:45.286501 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68965d-d8c2-44d4-b241-70c6a6f090e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:45.286786 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68965d-d8c2-44d4-b241-70c6a6f090e6 admin admin] PATH_INFO: `/v3/projects/a5d2c958ea024ee7be9e80a2af1392a7/users/e28781af3a244fe5ae8eddc43bdddbe9/roles/54195e660fc349978b39d5794765a095` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:45.288735 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1368/2741] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:12:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 16:12:45.301215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68965d-d8c2-44d4-b241-70c6a6f090e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5d2c958ea024ee7be9e80a2af1392a7, user_id=e28781af3a244fe5ae8eddc43bdddbe9, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:45.303973 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68965d-d8c2-44d4-b241-70c6a6f090e6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:45.318403 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1374/2742] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:12:45 2022] PUT /identity/v3/projects/a5d2c958ea024ee7be9e80a2af1392a7/users/e28781af3a244fe5ae8eddc43bdddbe9/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:12:45.323632 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d9fa5-6433-46eb-822f-d7e6719e9fc9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:45.323908 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d9fa5-6433-46eb-822f-d7e6719e9fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:45.324193 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d9fa5-6433-46eb-822f-d7e6719e9fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:45.342173 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9b9d9fa5-6433-46eb-822f-d7e6719e9fc9 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:12:45.343380 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9b9d9fa5-6433-46eb-822f-d7e6719e9fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:45.409338 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1369/2743] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:12:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:12:45.419184 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68965d-d8c2-44d4-b241-70c6a6f090e6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:45.420951 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0f68965d-d8c2-44d4-b241-70c6a6f090e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:45.475934 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d9fa5-6433-46eb-822f-d7e6719e9fc9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:45.477744 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9b9d9fa5-6433-46eb-822f-d7e6719e9fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:45.485643 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:12:45.487270 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68965d-d8c2-44d4-b241-70c6a6f090e6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:45.491688 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844297c-8fc0-4cfa-b778-9da58991bf38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:45.491867 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d9fa5-6433-46eb-822f-d7e6719e9fc9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:45.492352 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844297c-8fc0-4cfa-b778-9da58991bf38 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:45.492645 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844297c-8fc0-4cfa-b778-9da58991bf38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:45.492966 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844297c-8fc0-4cfa-b778-9da58991bf38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:45.498693 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd479b73-8d44-46db-979b-120a3992c74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:45.500094 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd479b73-8d44-46db-979b-120a3992c74d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:45.500871 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd479b73-8d44-46db-979b-120a3992c74d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:45.502161 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd479b73-8d44-46db-979b-120a3992c74d admin admin] PATH_INFO: `/v3/projects/5ad1ad798ef54f788b5eb3ad047c1836` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:45.502933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd479b73-8d44-46db-979b-120a3992c74d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ad1ad798ef54f788b5eb3ad047c1836)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:45.505118 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844297c-8fc0-4cfa-b778-9da58991bf38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:45.505397 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd479b73-8d44-46db-979b-120a3992c74d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:45.506611 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844297c-8fc0-4cfa-b778-9da58991bf38 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:45.516889 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1375/2744] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:12:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:12:45.559576 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cd479b73-8d44-46db-979b-120a3992c74d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ad1ad798ef54f788b5eb3ad047c1836', 'request_id': 'req-cd479b73-8d44-46db-979b-120a3992c74d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:45.566780 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cd479b73-8d44-46db-979b-120a3992c74d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ad1ad798ef54f788b5eb3ad047c1836', 'request_id': 'req-cd479b73-8d44-46db-979b-120a3992c74d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:45.573293 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cd479b73-8d44-46db-979b-120a3992c74d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ad1ad798ef54f788b5eb3ad047c1836', 'request_id': 'req-cd479b73-8d44-46db-979b-120a3992c74d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:45.585238 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cd479b73-8d44-46db-979b-120a3992c74d admin admin] The token cache is being invalidate because project 5ad1ad798ef54f788b5eb3ad047c1836 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:12:45.585607 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cd479b73-8d44-46db-979b-120a3992c74d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:45.586278 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cd479b73-8d44-46db-979b-120a3992c74d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:12:45.588212 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1370/2745] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:12:45 2022] DELETE /identity/v3/projects/5ad1ad798ef54f788b5eb3ad047c1836 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:12:45.593172 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0aab3-41c5-48e6-b305-53e86dc06019 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:45.593542 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0aab3-41c5-48e6-b305-53e86dc06019 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:45.593921 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0aab3-41c5-48e6-b305-53e86dc06019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:45.612744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2ea0aab3-41c5-48e6-b305-53e86dc06019 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:12:45.614613 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2ea0aab3-41c5-48e6-b305-53e86dc06019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:45.677512 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1376/2746] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:12:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:12:45.686071 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49533100-2e01-4156-bc6b-ce03b829ac9d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:45.686550 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49533100-2e01-4156-bc6b-ce03b829ac9d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:45.687035 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49533100-2e01-4156-bc6b-ce03b829ac9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:45.708315 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-49533100-2e01-4156-bc6b-ce03b829ac9d None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:12:45.709522 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-49533100-2e01-4156-bc6b-ce03b829ac9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:45.718403 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1371/2747] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:12:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:12:45.722916 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0aab3-41c5-48e6-b305-53e86dc06019 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:45.733205 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0aab3-41c5-48e6-b305-53e86dc06019 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:45.737915 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae03744e-0004-4d25-902e-83b992e04034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:45.738599 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae03744e-0004-4d25-902e-83b992e04034 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:45.738915 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae03744e-0004-4d25-902e-83b992e04034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:45.739201 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae03744e-0004-4d25-902e-83b992e04034 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:45.739931 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae03744e-0004-4d25-902e-83b992e04034 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:12:45.740240 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae03744e-0004-4d25-902e-83b992e04034 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:45.741849 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae03744e-0004-4d25-902e-83b992e04034 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:45.754618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1377/2748] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:12:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:12:45.759078 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49533100-2e01-4156-bc6b-ce03b829ac9d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:45.768625 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49533100-2e01-4156-bc6b-ce03b829ac9d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:45.776309 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97ec7c3-adad-40ff-a142-569edc321f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:45.777067 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ec7c3-adad-40ff-a142-569edc321f57 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:45.777419 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ec7c3-adad-40ff-a142-569edc321f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:45.777731 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ec7c3-adad-40ff-a142-569edc321f57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:45.780210 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97ec7c3-adad-40ff-a142-569edc321f57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:45.786901 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97ec7c3-adad-40ff-a142-569edc321f57 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:45.857015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1372/2749] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:12:45 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:12:45.861030 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae03744e-0004-4d25-902e-83b992e04034 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:45.872858 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae03744e-0004-4d25-902e-83b992e04034 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:45.968270 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc83e6b1-5e17-412d-b661-d0cce5c6b5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:45.969226 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83e6b1-5e17-412d-b661-d0cce5c6b5ec admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:45.969728 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83e6b1-5e17-412d-b661-d0cce5c6b5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:45.970100 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83e6b1-5e17-412d-b661-d0cce5c6b5ec admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:45.970988 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc83e6b1-5e17-412d-b661-d0cce5c6b5ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:45.974002 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc83e6b1-5e17-412d-b661-d0cce5c6b5ec admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:45.982001 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:12:45.999518 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1378/2750] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:12:45 2022] POST /identity/v3/users => generated 292 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:12:46.003439 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97ec7c3-adad-40ff-a142-569edc321f57 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:46.013659 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97ec7c3-adad-40ff-a142-569edc321f57 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:46.018304 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3b435-0b6f-48ed-bf98-7124b41ea3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:46.018980 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3b435-0b6f-48ed-bf98-7124b41ea3fb admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:46.019252 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3b435-0b6f-48ed-bf98-7124b41ea3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:46.019527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3b435-0b6f-48ed-bf98-7124b41ea3fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:46.020807 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e3b435-0b6f-48ed-bf98-7124b41ea3fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:46.022433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e3b435-0b6f-48ed-bf98-7124b41ea3fb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:46.033238 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1373/2751] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:12:46.037599 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc83e6b1-5e17-412d-b661-d0cce5c6b5ec admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:46.047767 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc83e6b1-5e17-412d-b661-d0cce5c6b5ec admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:46.052391 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ff9f0-bd66-4ab1-a391-2482cc471f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:46.053045 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ff9f0-bd66-4ab1-a391-2482cc471f9d admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:46.053331 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ff9f0-bd66-4ab1-a391-2482cc471f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:46.053629 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ff9f0-bd66-4ab1-a391-2482cc471f9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:46.054358 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ff9f0-bd66-4ab1-a391-2482cc471f9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:46.056053 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ff9f0-bd66-4ab1-a391-2482cc471f9d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:46.070007 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1379/2752] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:12:46.071280 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3b435-0b6f-48ed-bf98-7124b41ea3fb admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:46.083042 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3b435-0b6f-48ed-bf98-7124b41ea3fb admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:46.088037 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28837b77-bd41-4724-81fb-25a7d3a6708a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:46.088936 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28837b77-bd41-4724-81fb-25a7d3a6708a admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:46.089221 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28837b77-bd41-4724-81fb-25a7d3a6708a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:46.089503 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28837b77-bd41-4724-81fb-25a7d3a6708a admin admin] PATH_INFO: `/v3/projects/64e1c0ba9480420188d60da173aa1e45/users/c967025cedf94cc5a69ed5818a0fbd11/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:46.105076 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28837b77-bd41-4724-81fb-25a7d3a6708a admin admin] RBAC: Authorizing `identity:create_grant(project_id=64e1c0ba9480420188d60da173aa1e45, user_id=c967025cedf94cc5a69ed5818a0fbd11, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:46.107849 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28837b77-bd41-4724-81fb-25a7d3a6708a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:46.123045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1374/2753] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:12:46 2022] PUT /identity/v3/projects/64e1c0ba9480420188d60da173aa1e45/users/c967025cedf94cc5a69ed5818a0fbd11/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:12:46.128593 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4079a-235b-4c52-9fae-2e0cd3c775fb None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:46.129045 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4079a-235b-4c52-9fae-2e0cd3c775fb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:46.129387 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4079a-235b-4c52-9fae-2e0cd3c775fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:46.147550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-76b4079a-235b-4c52-9fae-2e0cd3c775fb None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:12:46.148777 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-76b4079a-235b-4c52-9fae-2e0cd3c775fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:46.214310 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1380/2754] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:12:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:12:46.224921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28837b77-bd41-4724-81fb-25a7d3a6708a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:46.227622 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-28837b77-bd41-4724-81fb-25a7d3a6708a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:46.307143 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28837b77-bd41-4724-81fb-25a7d3a6708a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:46.312013 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854ae27-2b17-4add-bd08-b6e0c7a466ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:46.313095 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854ae27-2b17-4add-bd08-b6e0c7a466ab service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:46.313444 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854ae27-2b17-4add-bd08-b6e0c7a466ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:46.313941 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854ae27-2b17-4add-bd08-b6e0c7a466ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:46.325623 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a854ae27-2b17-4add-bd08-b6e0c7a466ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:46.327096 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a854ae27-2b17-4add-bd08-b6e0c7a466ab service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:46.337764 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1375/2755] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:12:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:12:47.438958 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ac048-d0e7-4222-98a0-dd421db8a820 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:47.439499 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ac048-d0e7-4222-98a0-dd421db8a820 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:47.440090 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ac048-d0e7-4222-98a0-dd421db8a820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:47.472106 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ae0ac048-d0e7-4222-98a0-dd421db8a820 None None] MFA Rules not processed for user `c967025cedf94cc5a69ed5818a0fbd11`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:12:47.474030 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ae0ac048-d0e7-4222-98a0-dd421db8a820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:47.718966 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1381/2756] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:12:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:12:47.731292 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854ae27-2b17-4add-bd08-b6e0c7a466ab service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:47.734010 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a854ae27-2b17-4add-bd08-b6e0c7a466ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:47.771887 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:12:47.816234 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854ae27-2b17-4add-bd08-b6e0c7a466ab service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:47.820332 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08075f42-647e-4d4e-8645-9ad15f9bba8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:47.821665 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08075f42-647e-4d4e-8645-9ad15f9bba8c service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:47.821961 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08075f42-647e-4d4e-8645-9ad15f9bba8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:47.822245 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08075f42-647e-4d4e-8645-9ad15f9bba8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:47.849906 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08075f42-647e-4d4e-8645-9ad15f9bba8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:47.851654 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08075f42-647e-4d4e-8645-9ad15f9bba8c service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:47.863484 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1376/2757] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:12:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:12:49.471975 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ee82c-238f-4b84-b4cb-d19de9c92b09 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:49.473351 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ee82c-238f-4b84-b4cb-d19de9c92b09 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:49.473834 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ee82c-238f-4b84-b4cb-d19de9c92b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:49.504752 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-332ee82c-238f-4b84-b4cb-d19de9c92b09 None None] MFA Rules not processed for user `c967025cedf94cc5a69ed5818a0fbd11`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:12:49.506137 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-332ee82c-238f-4b84-b4cb-d19de9c92b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:49.515981 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1382/2758] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:12:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:12:50.679411 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b8b90-1112-4169-b250-a767f0e92668 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:50.679933 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b8b90-1112-4169-b250-a767f0e92668 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:50.680047 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b8b90-1112-4169-b250-a767f0e92668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:50.710231 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fc2b8b90-1112-4169-b250-a767f0e92668 None None] MFA Rules not processed for user `e28781af3a244fe5ae8eddc43bdddbe9`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:12:50.711471 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fc2b8b90-1112-4169-b250-a767f0e92668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:50.830478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1377/2759] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:12:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:12:50.836399 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ee82c-238f-4b84-b4cb-d19de9c92b09 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:50.838005 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-332ee82c-238f-4b84-b4cb-d19de9c92b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:50.857521 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ee82c-238f-4b84-b4cb-d19de9c92b09 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:50.868313 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609caac1-c7fb-4012-9647-f46bdc337b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:50.868660 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609caac1-c7fb-4012-9647-f46bdc337b5c admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:50.869127 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609caac1-c7fb-4012-9647-f46bdc337b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:50.869614 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609caac1-c7fb-4012-9647-f46bdc337b5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:50.871013 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609caac1-c7fb-4012-9647-f46bdc337b5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:50.873254 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609caac1-c7fb-4012-9647-f46bdc337b5c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:50.902982 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1383/2760] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:12:50 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:12:50.906983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b8b90-1112-4169-b250-a767f0e92668 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:50.924957 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b8b90-1112-4169-b250-a767f0e92668 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:51.053310 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c785019d-cfbf-4d13-8351-599fb551e778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:51.054354 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785019d-cfbf-4d13-8351-599fb551e778 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:51.054645 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785019d-cfbf-4d13-8351-599fb551e778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:51.054931 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785019d-cfbf-4d13-8351-599fb551e778 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:51.055684 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c785019d-cfbf-4d13-8351-599fb551e778 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:51.070238 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c785019d-cfbf-4d13-8351-599fb551e778 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:51.106253 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1378/2761] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:12:50 2022] POST /identity/v3/users => generated 299 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:12:51.113064 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609caac1-c7fb-4012-9647-f46bdc337b5c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:51.137781 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609caac1-c7fb-4012-9647-f46bdc337b5c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:51.149810 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad0fca-f001-417f-9d8c-b3f1611b961b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:51.150858 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad0fca-f001-417f-9d8c-b3f1611b961b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:51.151394 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad0fca-f001-417f-9d8c-b3f1611b961b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:51.151867 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad0fca-f001-417f-9d8c-b3f1611b961b admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:51.152810 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad0fca-f001-417f-9d8c-b3f1611b961b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:51.155724 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad0fca-f001-417f-9d8c-b3f1611b961b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:51.171043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1384/2762] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 16:12:51.174901 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c785019d-cfbf-4d13-8351-599fb551e778 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:51.188191 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c785019d-cfbf-4d13-8351-599fb551e778 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:51.193465 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a90f5c-63a0-4bdc-80d2-4893c28a9b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:51.194499 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a90f5c-63a0-4bdc-80d2-4893c28a9b6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:51.194894 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a90f5c-63a0-4bdc-80d2-4893c28a9b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:51.195188 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a90f5c-63a0-4bdc-80d2-4893c28a9b6b admin admin] PATH_INFO: `/v3/projects/114f7f29b73941ae83eef3fd58a6525f/users/6e016293b78e470e901c3b0e81de65f6/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:51.219092 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a90f5c-63a0-4bdc-80d2-4893c28a9b6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=114f7f29b73941ae83eef3fd58a6525f, user_id=6e016293b78e470e901c3b0e81de65f6, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:51.222540 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a90f5c-63a0-4bdc-80d2-4893c28a9b6b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:51.253615 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1379/2763] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:12:51 2022] PUT /identity/v3/projects/114f7f29b73941ae83eef3fd58a6525f/users/6e016293b78e470e901c3b0e81de65f6/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:12:51.256392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad0fca-f001-417f-9d8c-b3f1611b961b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:51.273757 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad0fca-f001-417f-9d8c-b3f1611b961b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:51.323099 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:12:51.362693 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b1100-4f7c-4945-9edd-b9c6533b47a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:51.364433 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b1100-4f7c-4945-9edd-b9c6533b47a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:51.365014 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b1100-4f7c-4945-9edd-b9c6533b47a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:51.365506 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b1100-4f7c-4945-9edd-b9c6533b47a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:51.366594 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022b1100-4f7c-4945-9edd-b9c6533b47a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:51.369666 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022b1100-4f7c-4945-9edd-b9c6533b47a3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:51.386091 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1385/2764] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:12:51.390157 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a90f5c-63a0-4bdc-80d2-4893c28a9b6b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:51.401831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a90f5c-63a0-4bdc-80d2-4893c28a9b6b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:51.408007 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26332cd5-7277-4777-bbe2-a93d6e3b00ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:51.408921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26332cd5-7277-4777-bbe2-a93d6e3b00ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:51.409209 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26332cd5-7277-4777-bbe2-a93d6e3b00ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:51.409495 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26332cd5-7277-4777-bbe2-a93d6e3b00ce admin admin] PATH_INFO: `/v3/projects/114f7f29b73941ae83eef3fd58a6525f/users/6e016293b78e470e901c3b0e81de65f6/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:51.412233 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26332cd5-7277-4777-bbe2-a93d6e3b00ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=114f7f29b73941ae83eef3fd58a6525f, user_id=6e016293b78e470e901c3b0e81de65f6, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:51.415006 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26332cd5-7277-4777-bbe2-a93d6e3b00ce admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:51.427665 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1380/2765] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:12:51 2022] PUT /identity/v3/projects/114f7f29b73941ae83eef3fd58a6525f/users/6e016293b78e470e901c3b0e81de65f6/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:12:57.326947 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af761e-334f-4f44-9a02-a0a9cdf48368 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:57.327862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af761e-334f-4f44-9a02-a0a9cdf48368 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:57.328312 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af761e-334f-4f44-9a02-a0a9cdf48368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:57.371918 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-23af761e-334f-4f44-9a02-a0a9cdf48368 None None] MFA Rules not processed for user `6e016293b78e470e901c3b0e81de65f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:12:57.373154 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-23af761e-334f-4f44-9a02-a0a9cdf48368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:57.489974 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1386/2766] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:12:57 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:12:57.510493 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26332cd5-7277-4777-bbe2-a93d6e3b00ce admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:57.512836 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-26332cd5-7277-4777-bbe2-a93d6e3b00ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:57.527827 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:12:57.630799 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26332cd5-7277-4777-bbe2-a93d6e3b00ce admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:57.634578 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d56f7-3e50-4847-8fe0-0fd8506a9969 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73a7d5f37e144768d7cc53f13cd5157', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:57.635243 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d56f7-3e50-4847-8fe0-0fd8506a9969 service glance] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:57.635543 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d56f7-3e50-4847-8fe0-0fd8506a9969 service glance] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:57.635808 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d56f7-3e50-4847-8fe0-0fd8506a9969 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:57.707512 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d56f7-3e50-4847-8fe0-0fd8506a9969 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:57.709345 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d56f7-3e50-4847-8fe0-0fd8506a9969 service glance] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:57.720399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1381/2767] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:12:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:12:57.843073 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b95a51-4d22-4aa8-bc11-15a1c6ebc90f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:57.843504 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b95a51-4d22-4aa8-bc11-15a1c6ebc90f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:57.844009 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b95a51-4d22-4aa8-bc11-15a1c6ebc90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:57.862447 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c6b95a51-4d22-4aa8-bc11-15a1c6ebc90f None None] MFA Rules not processed for user `6edbad98ed0f4823b9955f9f0e270149`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:12:57.864310 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c6b95a51-4d22-4aa8-bc11-15a1c6ebc90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:57.928026 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1387/2768] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 21 16:12:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:12:57.941050 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d56f7-3e50-4847-8fe0-0fd8506a9969 service glance] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:12:57.944127 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-340d56f7-3e50-4847-8fe0-0fd8506a9969 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:12:58.024897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d56f7-3e50-4847-8fe0-0fd8506a9969 service glance] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:12:58.030351 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b566421-6221-42a3-b4ba-5179b65475d0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5bfae2cb30843b1b1a0343efef38638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:12:58.031117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b566421-6221-42a3-b4ba-5179b65475d0 service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:12:58.031471 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b566421-6221-42a3-b4ba-5179b65475d0 service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:12:58.032021 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b566421-6221-42a3-b4ba-5179b65475d0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:12:58.043341 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b566421-6221-42a3-b4ba-5179b65475d0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:12:58.045170 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b566421-6221-42a3-b4ba-5179b65475d0 service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:12:58.055382 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1382/2769] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 16:12:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 16:13:07.920296 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=74 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:13:08.049666 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44862e58-ad4b-4b34-9080-560b2e2769da None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:13:08.049958 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44862e58-ad4b-4b34-9080-560b2e2769da None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:13:08.050243 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44862e58-ad4b-4b34-9080-560b2e2769da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:13:08.051770 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=76 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:13:08.070675 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-44862e58-ad4b-4b34-9080-560b2e2769da None None] MFA Rules not processed for user `e28781af3a244fe5ae8eddc43bdddbe9`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:13:08.072082 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-44862e58-ad4b-4b34-9080-560b2e2769da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:13:08.079674 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1388/2770] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:13:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:13:08.407819 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b566421-6221-42a3-b4ba-5179b65475d0 service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:13:08.409357 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2b566421-6221-42a3-b4ba-5179b65475d0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:13:08.470828 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b566421-6221-42a3-b4ba-5179b65475d0 service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:13:08.474989 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403fff7-46a9-4ee8-942d-1f5d8480b252 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:13:08.475524 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403fff7-46a9-4ee8-942d-1f5d8480b252 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:13:08.475809 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403fff7-46a9-4ee8-942d-1f5d8480b252 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:13:08.476089 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403fff7-46a9-4ee8-942d-1f5d8480b252 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:13:08.477961 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0403fff7-46a9-4ee8-942d-1f5d8480b252 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:13:08.545510 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403fff7-46a9-4ee8-942d-1f5d8480b252 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:13:08.547054 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403fff7-46a9-4ee8-942d-1f5d8480b252 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:13:08.558248 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1383/2771] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:13:08 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:13:10.401628 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368eb7f-ab46-449b-94b4-a10e6d26bcd2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:13:10.402110 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368eb7f-ab46-449b-94b4-a10e6d26bcd2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:13:10.403056 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368eb7f-ab46-449b-94b4-a10e6d26bcd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:13:10.432422 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0368eb7f-ab46-449b-94b4-a10e6d26bcd2 None None] MFA Rules not processed for user `6edbad98ed0f4823b9955f9f0e270149`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:13:10.433732 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0368eb7f-ab46-449b-94b4-a10e6d26bcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:13:10.457767 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1389/2772] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 21 16:13:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:13:10.458890 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403fff7-46a9-4ee8-942d-1f5d8480b252 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:13:10.485538 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403fff7-46a9-4ee8-942d-1f5d8480b252 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:13:10.491021 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46471f90-7771-4ee3-add4-369ab3f273ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5bfae2cb30843b1b1a0343efef38638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:13:10.491691 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46471f90-7771-4ee3-add4-369ab3f273ed service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:13:10.492146 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46471f90-7771-4ee3-add4-369ab3f273ed service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:13:10.492433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46471f90-7771-4ee3-add4-369ab3f273ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:13:10.503451 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46471f90-7771-4ee3-add4-369ab3f273ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:13:10.506082 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46471f90-7771-4ee3-add4-369ab3f273ed service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:13:10.517087 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1384/2773] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 16:13:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 16:13:20.421759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:13:20.512006 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=56 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:14:29.729179 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368eb7f-ab46-449b-94b4-a10e6d26bcd2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:14:29.738701 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368eb7f-ab46-449b-94b4-a10e6d26bcd2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:14:29.742201 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2195c-e7d4-4d62-8397-ba634d1d8c74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:14:29.742862 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2195c-e7d4-4d62-8397-ba634d1d8c74 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:14:29.743118 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2195c-e7d4-4d62-8397-ba634d1d8c74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:14:29.743398 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2195c-e7d4-4d62-8397-ba634d1d8c74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:14:29.745123 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-37f2195c-e7d4-4d62-8397-ba634d1d8c74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:14:29.826136 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2195c-e7d4-4d62-8397-ba634d1d8c74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:14:29.827602 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2195c-e7d4-4d62-8397-ba634d1d8c74 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:14:29.847757 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1390/2774] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:14:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:14:31.020632 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46471f90-7771-4ee3-add4-369ab3f273ed service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:14:31.030123 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46471f90-7771-4ee3-add4-369ab3f273ed service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:14:31.033658 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce4461-ac15-462c-9b4e-f40ec996f6e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:14:31.034289 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce4461-ac15-462c-9b4e-f40ec996f6e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:14:31.034574 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce4461-ac15-462c-9b4e-f40ec996f6e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:14:31.034853 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce4461-ac15-462c-9b4e-f40ec996f6e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:14:31.037130 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-62ce4461-ac15-462c-9b4e-f40ec996f6e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:14:31.050264 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce4461-ac15-462c-9b4e-f40ec996f6e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:14:31.052318 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce4461-ac15-462c-9b4e-f40ec996f6e8 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:14:31.063145 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1385/2775] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:14:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:14:36.113549 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2195c-e7d4-4d62-8397-ba634d1d8c74 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:14:36.122106 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2195c-e7d4-4d62-8397-ba634d1d8c74 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:14:36.125877 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d0b77-9cfb-4eff-8799-88ee539b390d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:14:36.126521 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d0b77-9cfb-4eff-8799-88ee539b390d service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:14:36.126806 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d0b77-9cfb-4eff-8799-88ee539b390d service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:14:36.127085 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d0b77-9cfb-4eff-8799-88ee539b390d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:14:36.137433 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d0b77-9cfb-4eff-8799-88ee539b390d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:14:36.139204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d0b77-9cfb-4eff-8799-88ee539b390d service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:14:36.157305 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1391/2776] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:14:36 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:14:41.058374 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:14:42.730161 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338d76a-dbe7-4be6-b1d0-0b21d28decbb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:14:42.730798 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338d76a-dbe7-4be6-b1d0-0b21d28decbb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:14:42.730798 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338d76a-dbe7-4be6-b1d0-0b21d28decbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:14:42.749460 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a338d76a-dbe7-4be6-b1d0-0b21d28decbb None None] MFA Rules not processed for user `805889a65fca41b38b334d7a7fdda4ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:14:42.750707 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a338d76a-dbe7-4be6-b1d0-0b21d28decbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:14:42.806579 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1386/2777] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:14:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:14:46.147850 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=49 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:14:52.721843 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d0b77-9cfb-4eff-8799-88ee539b390d service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:14:52.730690 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d0b77-9cfb-4eff-8799-88ee539b390d service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:14:52.734269 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bec900-278a-4529-8dc5-2a6cda052471 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:14:52.734897 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bec900-278a-4529-8dc5-2a6cda052471 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:14:52.735182 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bec900-278a-4529-8dc5-2a6cda052471 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:14:52.735457 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bec900-278a-4529-8dc5-2a6cda052471 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:14:52.737206 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b9bec900-278a-4529-8dc5-2a6cda052471 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:14:52.747023 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bec900-278a-4529-8dc5-2a6cda052471 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:14:52.748438 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bec900-278a-4529-8dc5-2a6cda052471 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:14:52.757054 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1392/2778] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:14:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:14:52.799167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=25 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:15:02.752833 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:15:06.031373 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338d76a-dbe7-4be6-b1d0-0b21d28decbb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:15:06.033626 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a338d76a-dbe7-4be6-b1d0-0b21d28decbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:15:06.053174 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338d76a-dbe7-4be6-b1d0-0b21d28decbb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:15:06.060343 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfad6bd-bfb7-4cc8-a7a2-5709eff1e81b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:15:06.060996 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfad6bd-bfb7-4cc8-a7a2-5709eff1e81b service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:15:06.061281 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfad6bd-bfb7-4cc8-a7a2-5709eff1e81b service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:15:06.061560 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfad6bd-bfb7-4cc8-a7a2-5709eff1e81b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:15:06.063361 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bbfad6bd-bfb7-4cc8-a7a2-5709eff1e81b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:15:06.077885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfad6bd-bfb7-4cc8-a7a2-5709eff1e81b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:15:06.079636 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfad6bd-bfb7-4cc8-a7a2-5709eff1e81b service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:15:06.090899 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1387/2779] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:15:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:15:16.086452 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:15:33.247674 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bec900-278a-4529-8dc5-2a6cda052471 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:15:33.249074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b9bec900-278a-4529-8dc5-2a6cda052471 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:15:33.264726 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bec900-278a-4529-8dc5-2a6cda052471 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:15:33.270212 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039cb31d-f995-4aa3-9398-ca749614a402 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:15:33.270848 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039cb31d-f995-4aa3-9398-ca749614a402 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:15:33.271133 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039cb31d-f995-4aa3-9398-ca749614a402 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:15:33.271412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039cb31d-f995-4aa3-9398-ca749614a402 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:15:33.273180 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-039cb31d-f995-4aa3-9398-ca749614a402 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:15:33.286226 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039cb31d-f995-4aa3-9398-ca749614a402 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:15:33.287806 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039cb31d-f995-4aa3-9398-ca749614a402 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:15:33.299245 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1393/2780] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:15:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 16:15:43.292821 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:15:45.752992 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfad6bd-bfb7-4cc8-a7a2-5709eff1e81b service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:15:45.767373 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfad6bd-bfb7-4cc8-a7a2-5709eff1e81b service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:15:45.786774 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8de1a-d272-4d11-bdaa-4ad746efb334 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:15:45.789445 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8de1a-d272-4d11-bdaa-4ad746efb334 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:15:45.789896 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8de1a-d272-4d11-bdaa-4ad746efb334 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:15:45.790180 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8de1a-d272-4d11-bdaa-4ad746efb334 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:15:45.793432 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-46b8de1a-d272-4d11-bdaa-4ad746efb334 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:15:45.817030 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b8de1a-d272-4d11-bdaa-4ad746efb334 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:15:45.820380 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b8de1a-d272-4d11-bdaa-4ad746efb334 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:15:45.836007 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1388/2781] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:15:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:15:55.829180 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:17:32.308017 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039cb31d-f995-4aa3-9398-ca749614a402 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:17:32.319792 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039cb31d-f995-4aa3-9398-ca749614a402 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:17:32.325194 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0294d-fa76-4ad2-a3a0-15fe1a95fa2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:17:32.326049 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0294d-fa76-4ad2-a3a0-15fe1a95fa2a service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:17:32.326275 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0294d-fa76-4ad2-a3a0-15fe1a95fa2a service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:17:32.326509 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0294d-fa76-4ad2-a3a0-15fe1a95fa2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:17:32.328279 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5cd0294d-fa76-4ad2-a3a0-15fe1a95fa2a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:17:32.410498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd0294d-fa76-4ad2-a3a0-15fe1a95fa2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:17:32.412011 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd0294d-fa76-4ad2-a3a0-15fe1a95fa2a service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:17:32.430041 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1394/2782] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:17:32 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:17:42.427391 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=41 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:17:47.963613 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8de1a-d272-4d11-bdaa-4ad746efb334 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:17:47.965064 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-46b8de1a-d272-4d11-bdaa-4ad746efb334 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:17:48.042369 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8de1a-d272-4d11-bdaa-4ad746efb334 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:17:48.045949 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41901ac0-f5ac-4d51-ad6f-bbca995077cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:17:48.046591 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41901ac0-f5ac-4d51-ad6f-bbca995077cf service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:17:48.046873 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41901ac0-f5ac-4d51-ad6f-bbca995077cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:17:48.047148 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41901ac0-f5ac-4d51-ad6f-bbca995077cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:17:48.105164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41901ac0-f5ac-4d51-ad6f-bbca995077cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:17:48.106661 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41901ac0-f5ac-4d51-ad6f-bbca995077cf service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:17:48.115982 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1389/2783] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:17:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:17:57.820843 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0294d-fa76-4ad2-a3a0-15fe1a95fa2a service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:17:57.832689 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0294d-fa76-4ad2-a3a0-15fe1a95fa2a service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:17:57.838286 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68976c3e-c147-47e7-b8d8-6ac75504118d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:17:57.838953 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68976c3e-c147-47e7-b8d8-6ac75504118d service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:17:57.839238 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68976c3e-c147-47e7-b8d8-6ac75504118d service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:17:57.839520 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68976c3e-c147-47e7-b8d8-6ac75504118d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:17:57.848987 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68976c3e-c147-47e7-b8d8-6ac75504118d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:17:57.850626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68976c3e-c147-47e7-b8d8-6ac75504118d service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:17:57.861286 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1395/2784] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:17:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:17:58.111494 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=57 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:18:07.857192 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:18:36.832663 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41901ac0-f5ac-4d51-ad6f-bbca995077cf service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:18:36.834087 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-41901ac0-f5ac-4d51-ad6f-bbca995077cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:18:36.903667 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41901ac0-f5ac-4d51-ad6f-bbca995077cf service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:18:36.910250 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8497032-708d-473f-bd4a-8422e24719e7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822d9d3521db4a9688f51c6313c7dd74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:18:36.910908 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8497032-708d-473f-bd4a-8422e24719e7 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:18:36.911194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8497032-708d-473f-bd4a-8422e24719e7 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:18:36.911472 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8497032-708d-473f-bd4a-8422e24719e7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:18:36.913383 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b8497032-708d-473f-bd4a-8422e24719e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:18:36.925921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8497032-708d-473f-bd4a-8422e24719e7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:18:36.927513 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8497032-708d-473f-bd4a-8422e24719e7 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:18:36.938061 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1390/2785] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:18:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:18:46.933821 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=32 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:19:29.278113 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68976c3e-c147-47e7-b8d8-6ac75504118d service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:19:29.288486 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68976c3e-c147-47e7-b8d8-6ac75504118d service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:19:29.294834 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60306a57-a5e7-4b28-bbff-0f6b782524cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:19:29.295805 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60306a57-a5e7-4b28-bbff-0f6b782524cc service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:19:29.296114 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60306a57-a5e7-4b28-bbff-0f6b782524cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:19:29.296480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60306a57-a5e7-4b28-bbff-0f6b782524cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:19:29.310329 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60306a57-a5e7-4b28-bbff-0f6b782524cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:19:29.312432 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60306a57-a5e7-4b28-bbff-0f6b782524cc service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:19:29.327437 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1396/2786] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:19:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:19:39.320964 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:20:49.552893 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8497032-708d-473f-bd4a-8422e24719e7 service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:20:49.562375 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8497032-708d-473f-bd4a-8422e24719e7 service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:20:49.565881 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac7bf3e-2c1a-4fee-8d87-7a051897d173 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:20:49.566508 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7bf3e-2c1a-4fee-8d87-7a051897d173 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:20:49.566789 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7bf3e-2c1a-4fee-8d87-7a051897d173 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:20:49.567060 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7bf3e-2c1a-4fee-8d87-7a051897d173 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:20:49.574890 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac7bf3e-2c1a-4fee-8d87-7a051897d173 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:20:49.576326 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac7bf3e-2c1a-4fee-8d87-7a051897d173 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:20:49.584355 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1391/2787] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:20:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:20:59.580909 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:21:06.456932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60306a57-a5e7-4b28-bbff-0f6b782524cc service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:21:06.467165 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60306a57-a5e7-4b28-bbff-0f6b782524cc service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:21:06.471051 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b1480-1988-4198-a34d-f1af9ad83d0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:21:06.471894 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b1480-1988-4198-a34d-f1af9ad83d0d service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:21:06.472237 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b1480-1988-4198-a34d-f1af9ad83d0d service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:21:06.472577 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b1480-1988-4198-a34d-f1af9ad83d0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:21:06.485007 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b1480-1988-4198-a34d-f1af9ad83d0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:21:06.487039 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b1480-1988-4198-a34d-f1af9ad83d0d service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:21:06.498083 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1397/2788] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:21:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:21:06.547797 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac7bf3e-2c1a-4fee-8d87-7a051897d173 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:21:06.557470 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac7bf3e-2c1a-4fee-8d87-7a051897d173 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:21:06.561095 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29550e11-e9e1-44f9-83c4-da06a8e36e12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:21:06.561780 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29550e11-e9e1-44f9-83c4-da06a8e36e12 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:21:06.562071 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29550e11-e9e1-44f9-83c4-da06a8e36e12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:21:06.562349 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29550e11-e9e1-44f9-83c4-da06a8e36e12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:21:06.571403 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29550e11-e9e1-44f9-83c4-da06a8e36e12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:21:06.573351 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29550e11-e9e1-44f9-83c4-da06a8e36e12 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:21:06.600841 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1392/2789] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:21:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 16:21:16.492872 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:21:16.598082 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=13 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:23:06.765714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b1480-1988-4198-a34d-f1af9ad83d0d service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:23:06.767295 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d04b1480-1988-4198-a34d-f1af9ad83d0d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:23:06.808650 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b1480-1988-4198-a34d-f1af9ad83d0d service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:23:06.812763 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6a67b-8df2-438d-961c-85a31a24edb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:23:06.813415 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6a67b-8df2-438d-961c-85a31a24edb7 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:23:06.813733 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6a67b-8df2-438d-961c-85a31a24edb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:23:06.814009 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6a67b-8df2-438d-961c-85a31a24edb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:23:06.836972 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe6a67b-8df2-438d-961c-85a31a24edb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:23:06.838445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe6a67b-8df2-438d-961c-85a31a24edb7 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:23:06.848118 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1398/2790] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:23:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 16:23:06.895579 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29550e11-e9e1-44f9-83c4-da06a8e36e12 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:23:06.915566 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29550e11-e9e1-44f9-83c4-da06a8e36e12 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:23:06.919924 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c432c1-f3cd-4931-afb2-b86cf372f1db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:23:06.920574 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c432c1-f3cd-4931-afb2-b86cf372f1db service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:23:06.920857 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c432c1-f3cd-4931-afb2-b86cf372f1db service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:23:06.921136 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c432c1-f3cd-4931-afb2-b86cf372f1db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:23:06.923250 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b4c432c1-f3cd-4931-afb2-b86cf372f1db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:23:06.965366 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c432c1-f3cd-4931-afb2-b86cf372f1db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:23:06.966844 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c432c1-f3cd-4931-afb2-b86cf372f1db service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:23:06.986148 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1393/2791] 10.222.0.52 () {60 vars in 1396 bytes} [Thu Jul 21 16:23:06 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:23:07.037043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6a67b-8df2-438d-961c-85a31a24edb7 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:23:07.047943 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6a67b-8df2-438d-961c-85a31a24edb7 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:23:07.051757 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e696e-262b-4b0a-a4ce-c1330396dda2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:23:07.052412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e696e-262b-4b0a-a4ce-c1330396dda2 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:23:07.052701 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e696e-262b-4b0a-a4ce-c1330396dda2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:23:07.053010 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e696e-262b-4b0a-a4ce-c1330396dda2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:23:07.055123 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f01e696e-262b-4b0a-a4ce-c1330396dda2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:23:07.142309 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01e696e-262b-4b0a-a4ce-c1330396dda2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:23:07.143794 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01e696e-262b-4b0a-a4ce-c1330396dda2 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:23:07.162888 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1399/2792] 10.222.0.52 () {60 vars in 1396 bytes} [Thu Jul 21 16:23:07 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3252 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 16:23:16.983210 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=24 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:23:17.160141 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=66 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:23:38.865885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c432c1-f3cd-4931-afb2-b86cf372f1db service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:23:38.899089 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c432c1-f3cd-4931-afb2-b86cf372f1db service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:23:38.904192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16095e0-2763-4058-acd3-6d58af4c73b6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822d9d3521db4a9688f51c6313c7dd74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:23:38.904842 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16095e0-2763-4058-acd3-6d58af4c73b6 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:23:38.905133 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16095e0-2763-4058-acd3-6d58af4c73b6 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:23:38.905414 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16095e0-2763-4058-acd3-6d58af4c73b6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:23:38.920132 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16095e0-2763-4058-acd3-6d58af4c73b6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:23:38.921777 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16095e0-2763-4058-acd3-6d58af4c73b6 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:23:38.933893 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1394/2793] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:23:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:23:48.604222 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e696e-262b-4b0a-a4ce-c1330396dda2 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:23:48.627826 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e696e-262b-4b0a-a4ce-c1330396dda2 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:23:48.641357 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d488cb0c-a792-4b53-af9b-4eb14cd136b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:23:48.642308 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488cb0c-a792-4b53-af9b-4eb14cd136b1 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:23:48.642597 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488cb0c-a792-4b53-af9b-4eb14cd136b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:23:48.642882 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488cb0c-a792-4b53-af9b-4eb14cd136b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:23:48.653279 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d488cb0c-a792-4b53-af9b-4eb14cd136b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:23:48.654951 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d488cb0c-a792-4b53-af9b-4eb14cd136b1 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:23:48.666288 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1400/2794] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:23:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:23:48.929081 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=13 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:23:58.661681 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=9 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:25:07.281153 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16095e0-2763-4058-acd3-6d58af4c73b6 service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:25:07.292313 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16095e0-2763-4058-acd3-6d58af4c73b6 service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:25:07.295840 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468f9bd-d4d0-4675-b586-bcd7dcf7b0bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:25:07.296484 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468f9bd-d4d0-4675-b586-bcd7dcf7b0bb service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:25:07.296767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468f9bd-d4d0-4675-b586-bcd7dcf7b0bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:25:07.297045 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468f9bd-d4d0-4675-b586-bcd7dcf7b0bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:25:07.299037 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d468f9bd-d4d0-4675-b586-bcd7dcf7b0bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:25:07.401738 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d468f9bd-d4d0-4675-b586-bcd7dcf7b0bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:25:07.404009 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d468f9bd-d4d0-4675-b586-bcd7dcf7b0bb service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:25:07.429920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1395/2795] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:25:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:25:17.427144 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=43 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:25:53.540791 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d488cb0c-a792-4b53-af9b-4eb14cd136b1 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:25:53.542587 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d488cb0c-a792-4b53-af9b-4eb14cd136b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:25:53.624827 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d488cb0c-a792-4b53-af9b-4eb14cd136b1 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:25:53.628519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37ba551-5979-4399-9ca5-3af43a95081e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:25:53.629165 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ba551-5979-4399-9ca5-3af43a95081e service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:25:53.629451 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ba551-5979-4399-9ca5-3af43a95081e service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:25:53.629772 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ba551-5979-4399-9ca5-3af43a95081e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:25:53.631702 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b37ba551-5979-4399-9ca5-3af43a95081e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:25:53.642346 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37ba551-5979-4399-9ca5-3af43a95081e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:25:53.643821 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37ba551-5979-4399-9ca5-3af43a95081e service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:25:53.652204 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1401/2796] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:25:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:26:03.648415 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=35 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:26:06.807588 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468f9bd-d4d0-4675-b586-bcd7dcf7b0bb service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:26:06.816630 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468f9bd-d4d0-4675-b586-bcd7dcf7b0bb service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:26:06.820314 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac23ed1-5c58-40c3-b16c-7aeeb851c63c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:26:06.820961 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac23ed1-5c58-40c3-b16c-7aeeb851c63c service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:26:06.821245 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac23ed1-5c58-40c3-b16c-7aeeb851c63c service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:26:06.821633 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac23ed1-5c58-40c3-b16c-7aeeb851c63c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:26:06.823624 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cac23ed1-5c58-40c3-b16c-7aeeb851c63c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:26:06.900057 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac23ed1-5c58-40c3-b16c-7aeeb851c63c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:26:06.901782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac23ed1-5c58-40c3-b16c-7aeeb851c63c service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:26:06.911919 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1396/2797] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:26:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:26:16.907447 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=33 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:28:06.042864 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37ba551-5979-4399-9ca5-3af43a95081e service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:28:06.044445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b37ba551-5979-4399-9ca5-3af43a95081e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:28:06.124848 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37ba551-5979-4399-9ca5-3af43a95081e service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:28:06.128535 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a325d25-1f68-4e3d-a48f-60b2dd9d19e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:28:06.129184 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a325d25-1f68-4e3d-a48f-60b2dd9d19e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:28:06.129473 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a325d25-1f68-4e3d-a48f-60b2dd9d19e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:28:06.129777 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a325d25-1f68-4e3d-a48f-60b2dd9d19e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:28:06.139540 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a325d25-1f68-4e3d-a48f-60b2dd9d19e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:28:06.140914 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a325d25-1f68-4e3d-a48f-60b2dd9d19e7 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:28:06.150011 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1402/2798] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:28:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:28:06.200634 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac23ed1-5c58-40c3-b16c-7aeeb851c63c service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:28:06.266832 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac23ed1-5c58-40c3-b16c-7aeeb851c63c service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:28:06.270616 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e7586-b5ce-4669-9e7a-55109f320391 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:28:06.271289 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e7586-b5ce-4669-9e7a-55109f320391 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:28:06.271601 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e7586-b5ce-4669-9e7a-55109f320391 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:28:06.271937 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e7586-b5ce-4669-9e7a-55109f320391 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:28:06.273908 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-148e7586-b5ce-4669-9e7a-55109f320391 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:28:06.357428 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e7586-b5ce-4669-9e7a-55109f320391 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:28:06.359115 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e7586-b5ce-4669-9e7a-55109f320391 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:28:06.377944 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1397/2799] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:28:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:28:06.960577 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a325d25-1f68-4e3d-a48f-60b2dd9d19e7 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:28:06.969937 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a325d25-1f68-4e3d-a48f-60b2dd9d19e7 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:28:06.973433 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c815478-72d0-4d36-ae19-f45a516bd8f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:28:06.974121 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c815478-72d0-4d36-ae19-f45a516bd8f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:28:06.974402 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c815478-72d0-4d36-ae19-f45a516bd8f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:28:06.974676 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c815478-72d0-4d36-ae19-f45a516bd8f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:28:07.031293 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c815478-72d0-4d36-ae19-f45a516bd8f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:28:07.032746 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c815478-72d0-4d36-ae19-f45a516bd8f5 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:28:07.041914 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1403/2800] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:28:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:28:16.375277 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=65 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:28:17.037658 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=63 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:28:44.864108 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e7586-b5ce-4669-9e7a-55109f320391 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:28:44.865725 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-148e7586-b5ce-4669-9e7a-55109f320391 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:28:44.948419 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e7586-b5ce-4669-9e7a-55109f320391 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:28:44.954101 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09312541-b580-4923-8f68-5f465fc9c896 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822d9d3521db4a9688f51c6313c7dd74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:28:44.954746 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09312541-b580-4923-8f68-5f465fc9c896 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:28:44.955031 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09312541-b580-4923-8f68-5f465fc9c896 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:28:44.955309 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09312541-b580-4923-8f68-5f465fc9c896 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:28:44.957209 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-09312541-b580-4923-8f68-5f465fc9c896 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:28:44.969068 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09312541-b580-4923-8f68-5f465fc9c896 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:28:44.970737 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09312541-b580-4923-8f68-5f465fc9c896 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:28:44.980759 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1398/2801] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:28:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:28:54.976685 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=37 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:30:56.516692 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c815478-72d0-4d36-ae19-f45a516bd8f5 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:30:56.526319 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c815478-72d0-4d36-ae19-f45a516bd8f5 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:30:56.529998 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1741cfb-57c6-432a-808b-05e1ec111e44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:30:56.530651 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1741cfb-57c6-432a-808b-05e1ec111e44 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:30:56.530939 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1741cfb-57c6-432a-808b-05e1ec111e44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:30:56.531219 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1741cfb-57c6-432a-808b-05e1ec111e44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:30:56.539509 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1741cfb-57c6-432a-808b-05e1ec111e44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:30:56.540969 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1741cfb-57c6-432a-808b-05e1ec111e44 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:30:56.549714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1404/2802] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:30:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:31:06.545413 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:31:07.691568 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09312541-b580-4923-8f68-5f465fc9c896 service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:31:07.701297 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09312541-b580-4923-8f68-5f465fc9c896 service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:31:07.705137 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f863b-4f07-471a-a728-c8ec55b6bee6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:31:07.705875 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f863b-4f07-471a-a728-c8ec55b6bee6 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:31:07.706227 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f863b-4f07-471a-a728-c8ec55b6bee6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:31:07.706558 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f863b-4f07-471a-a728-c8ec55b6bee6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:31:07.716101 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f863b-4f07-471a-a728-c8ec55b6bee6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:31:07.717558 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f863b-4f07-471a-a728-c8ec55b6bee6 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:31:07.748295 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1399/2803] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:31:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:31:08.929648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1741cfb-57c6-432a-808b-05e1ec111e44 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:31:08.932320 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d1741cfb-57c6-432a-808b-05e1ec111e44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:31:08.947406 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1741cfb-57c6-432a-808b-05e1ec111e44 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:31:08.952206 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0924c-e84e-4f3c-bdae-9794aa51b542 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:31:08.952863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0924c-e84e-4f3c-bdae-9794aa51b542 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:31:08.953158 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0924c-e84e-4f3c-bdae-9794aa51b542 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:31:08.953445 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0924c-e84e-4f3c-bdae-9794aa51b542 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:31:08.971833 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b0924c-e84e-4f3c-bdae-9794aa51b542 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:31:08.973629 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b0924c-e84e-4f3c-bdae-9794aa51b542 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:31:08.992206 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1405/2804] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:31:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:31:09.042687 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f863b-4f07-471a-a728-c8ec55b6bee6 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:31:09.053990 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f863b-4f07-471a-a728-c8ec55b6bee6 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:31:09.057739 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad112ce7-915d-4547-a6c0-af9b2e440152 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:31:09.058389 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad112ce7-915d-4547-a6c0-af9b2e440152 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:31:09.058683 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad112ce7-915d-4547-a6c0-af9b2e440152 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:31:09.058967 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad112ce7-915d-4547-a6c0-af9b2e440152 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:31:09.060953 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ad112ce7-915d-4547-a6c0-af9b2e440152 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:31:09.076575 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad112ce7-915d-4547-a6c0-af9b2e440152 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:31:09.078128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad112ce7-915d-4547-a6c0-af9b2e440152 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:31:09.091829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1400/2805] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:31:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:31:10.101527 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0924c-e84e-4f3c-bdae-9794aa51b542 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:31:10.103327 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-78b0924c-e84e-4f3c-bdae-9794aa51b542 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:31:10.121163 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0924c-e84e-4f3c-bdae-9794aa51b542 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:31:10.126881 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244791e5-8470-4795-b8fb-ff9040d8c782 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:31:10.127797 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244791e5-8470-4795-b8fb-ff9040d8c782 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:31:10.128091 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244791e5-8470-4795-b8fb-ff9040d8c782 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:31:10.128374 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244791e5-8470-4795-b8fb-ff9040d8c782 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:31:10.130310 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-244791e5-8470-4795-b8fb-ff9040d8c782 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:31:10.143818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244791e5-8470-4795-b8fb-ff9040d8c782 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:31:10.145537 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244791e5-8470-4795-b8fb-ff9040d8c782 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:31:10.157034 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1406/2806] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:31:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:31:19.086348 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=19 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:31:20.152240 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=12 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:33:07.232634 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad112ce7-915d-4547-a6c0-af9b2e440152 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:33:07.252983 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad112ce7-915d-4547-a6c0-af9b2e440152 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:33:07.256611 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9291fd1-2264-4e8e-952b-df675b4fff5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:33:07.257244 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9291fd1-2264-4e8e-952b-df675b4fff5b service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:33:07.257528 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9291fd1-2264-4e8e-952b-df675b4fff5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:33:07.257855 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9291fd1-2264-4e8e-952b-df675b4fff5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:33:07.259731 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b9291fd1-2264-4e8e-952b-df675b4fff5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:33:07.317951 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9291fd1-2264-4e8e-952b-df675b4fff5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:33:07.319414 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9291fd1-2264-4e8e-952b-df675b4fff5b service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:33:07.338301 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1401/2807] 10.222.0.52 () {60 vars in 1396 bytes} [Thu Jul 21 16:33:07 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3250 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:33:07.453300 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244791e5-8470-4795-b8fb-ff9040d8c782 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:33:07.455394 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-244791e5-8470-4795-b8fb-ff9040d8c782 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:33:07.520828 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244791e5-8470-4795-b8fb-ff9040d8c782 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:33:07.527015 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6246eb-7a6f-40b4-98fc-8285eefbf4a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:33:07.527698 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6246eb-7a6f-40b4-98fc-8285eefbf4a5 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:33:07.527955 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6246eb-7a6f-40b4-98fc-8285eefbf4a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:33:07.528235 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6246eb-7a6f-40b4-98fc-8285eefbf4a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:33:07.530168 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fe6246eb-7a6f-40b4-98fc-8285eefbf4a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:33:07.611624 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6246eb-7a6f-40b4-98fc-8285eefbf4a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:33:07.613249 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6246eb-7a6f-40b4-98fc-8285eefbf4a5 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:33:07.632670 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1407/2808] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:33:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:33:10.206213 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9291fd1-2264-4e8e-952b-df675b4fff5b service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:33:10.207869 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b9291fd1-2264-4e8e-952b-df675b4fff5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:33:10.219569 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9291fd1-2264-4e8e-952b-df675b4fff5b service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:33:10.223208 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6bccd-4f5b-44c5-8c54-10eb33edba4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:33:10.223869 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6bccd-4f5b-44c5-8c54-10eb33edba4e service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:33:10.224150 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6bccd-4f5b-44c5-8c54-10eb33edba4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:33:10.224424 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6bccd-4f5b-44c5-8c54-10eb33edba4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:33:10.234925 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e6bccd-4f5b-44c5-8c54-10eb33edba4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:33:10.236856 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e6bccd-4f5b-44c5-8c54-10eb33edba4e service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:33:10.254897 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1402/2809] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:33:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:33:17.629793 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=60 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:33:20.246611 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=40 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:33:47.836314 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6246eb-7a6f-40b4-98fc-8285eefbf4a5 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:33:47.837741 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fe6246eb-7a6f-40b4-98fc-8285eefbf4a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:33:47.869345 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6246eb-7a6f-40b4-98fc-8285eefbf4a5 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:33:47.874320 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef81920-3cd4-4893-9f4a-de791d75c785 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822d9d3521db4a9688f51c6313c7dd74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:33:47.874958 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef81920-3cd4-4893-9f4a-de791d75c785 service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:33:47.875242 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef81920-3cd4-4893-9f4a-de791d75c785 service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:33:47.875519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef81920-3cd4-4893-9f4a-de791d75c785 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:33:47.884046 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef81920-3cd4-4893-9f4a-de791d75c785 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:33:47.885633 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef81920-3cd4-4893-9f4a-de791d75c785 service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:33:47.896507 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1408/2810] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:33:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:33:57.891735 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=13 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:35:12.783273 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6bccd-4f5b-44c5-8c54-10eb33edba4e service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:35:12.872678 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6bccd-4f5b-44c5-8c54-10eb33edba4e service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:35:12.878260 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e0a35-ea03-42df-a642-ec841d297dcb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:35:12.878907 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e0a35-ea03-42df-a642-ec841d297dcb service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:35:12.879190 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e0a35-ea03-42df-a642-ec841d297dcb service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:35:12.879465 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e0a35-ea03-42df-a642-ec841d297dcb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:35:12.881353 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-425e0a35-ea03-42df-a642-ec841d297dcb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:35:12.907011 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e0a35-ea03-42df-a642-ec841d297dcb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:35:12.908646 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e0a35-ea03-42df-a642-ec841d297dcb service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:35:12.920662 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1403/2811] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:35:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:35:22.915996 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=46 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:36:09.212626 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef81920-3cd4-4893-9f4a-de791d75c785 service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:36:09.278293 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef81920-3cd4-4893-9f4a-de791d75c785 service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:36:09.282011 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac3c7c-e07d-4cc7-9e2d-bcd2291079da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:36:09.282657 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac3c7c-e07d-4cc7-9e2d-bcd2291079da service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:36:09.282946 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac3c7c-e07d-4cc7-9e2d-bcd2291079da service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:36:09.283228 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac3c7c-e07d-4cc7-9e2d-bcd2291079da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:36:09.293642 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac3c7c-e07d-4cc7-9e2d-bcd2291079da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:36:09.295070 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac3c7c-e07d-4cc7-9e2d-bcd2291079da service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:36:09.306063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1409/2812] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:36:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:36:19.301172 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=30 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:37:09.092120 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff4497-6972-4f43-a15b-77e3a2c43bc8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:37:09.092661 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff4497-6972-4f43-a15b-77e3a2c43bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:37:09.092831 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff4497-6972-4f43-a15b-77e3a2c43bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:37:09.154788 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7bff4497-6972-4f43-a15b-77e3a2c43bc8 None None] MFA Rules not processed for user `6edbad98ed0f4823b9955f9f0e270149`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:37:09.155900 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7bff4497-6972-4f43-a15b-77e3a2c43bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:37:09.229722 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1404/2813] 10.222.0.52 () {58 vars in 967 bytes} [Thu Jul 21 16:37:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:37:09.276253 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac3c7c-e07d-4cc7-9e2d-bcd2291079da service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:37:09.278108 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e1ac3c7c-e07d-4cc7-9e2d-bcd2291079da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:37:09.374115 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac3c7c-e07d-4cc7-9e2d-bcd2291079da service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:37:09.379636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a65458-bba6-49d0-bba0-7ebfb04c869c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5bfae2cb30843b1b1a0343efef38638', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:37:09.380286 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a65458-bba6-49d0-bba0-7ebfb04c869c service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:37:09.380573 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a65458-bba6-49d0-bba0-7ebfb04c869c service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:37:09.380851 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a65458-bba6-49d0-bba0-7ebfb04c869c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:37:09.389753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a65458-bba6-49d0-bba0-7ebfb04c869c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:37:09.391352 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a65458-bba6-49d0-bba0-7ebfb04c869c service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:37:09.399730 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1410/2814] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 16:37:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 16:37:09.978620 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bff4497-6972-4f43-a15b-77e3a2c43bc8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:37:09.991316 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bff4497-6972-4f43-a15b-77e3a2c43bc8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:37:09.995088 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34e0ed-16ac-4c2d-95e2-fc7d18cdf8aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:37:09.995754 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34e0ed-16ac-4c2d-95e2-fc7d18cdf8aa service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:37:09.996039 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34e0ed-16ac-4c2d-95e2-fc7d18cdf8aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:37:09.996396 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34e0ed-16ac-4c2d-95e2-fc7d18cdf8aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:37:09.998339 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ba34e0ed-16ac-4c2d-95e2-fc7d18cdf8aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:37:10.100652 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba34e0ed-16ac-4c2d-95e2-fc7d18cdf8aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:37:10.102238 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba34e0ed-16ac-4c2d-95e2-fc7d18cdf8aa service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:37:10.121888 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1405/2815] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:37:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:37:10.508782 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a65458-bba6-49d0-bba0-7ebfb04c869c service swift] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:37:10.518392 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a65458-bba6-49d0-bba0-7ebfb04c869c service swift] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:37:10.521977 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7a24c-7b2d-432d-bb0a-d1ebfe1c5a1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:37:10.522612 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7a24c-7b2d-432d-bb0a-d1ebfe1c5a1f service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:37:10.522899 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7a24c-7b2d-432d-bb0a-d1ebfe1c5a1f service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:37:10.523177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7a24c-7b2d-432d-bb0a-d1ebfe1c5a1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:37:10.525052 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2fe7a24c-7b2d-432d-bb0a-d1ebfe1c5a1f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:37:10.601688 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe7a24c-7b2d-432d-bb0a-d1ebfe1c5a1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:37:10.603083 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe7a24c-7b2d-432d-bb0a-d1ebfe1c5a1f service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:37:10.613283 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1411/2816] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:37:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:37:13.278432 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34e0ed-16ac-4c2d-95e2-fc7d18cdf8aa service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:37:13.290197 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34e0ed-16ac-4c2d-95e2-fc7d18cdf8aa service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:37:13.295153 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc7de1c-6d3c-4a99-a762-ae1aec9867f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822d9d3521db4a9688f51c6313c7dd74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:37:13.296295 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7de1c-6d3c-4a99-a762-ae1aec9867f0 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:37:13.296567 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7de1c-6d3c-4a99-a762-ae1aec9867f0 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:37:13.296839 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7de1c-6d3c-4a99-a762-ae1aec9867f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:37:13.298676 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cfc7de1c-6d3c-4a99-a762-ae1aec9867f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:37:13.310353 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc7de1c-6d3c-4a99-a762-ae1aec9867f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:37:13.312630 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc7de1c-6d3c-4a99-a762-ae1aec9867f0 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:37:13.323509 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1406/2817] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:37:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:37:16.492554 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7a24c-7b2d-432d-bb0a-d1ebfe1c5a1f service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:37:16.533438 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7a24c-7b2d-432d-bb0a-d1ebfe1c5a1f service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:37:16.541550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36908480-1558-4799-84d9-d1255e8f2c92 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822d9d3521db4a9688f51c6313c7dd74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:37:16.542350 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36908480-1558-4799-84d9-d1255e8f2c92 service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:37:16.542636 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36908480-1558-4799-84d9-d1255e8f2c92 service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:37:16.542916 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36908480-1558-4799-84d9-d1255e8f2c92 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:37:16.545080 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-36908480-1558-4799-84d9-d1255e8f2c92 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:37:16.564006 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36908480-1558-4799-84d9-d1255e8f2c92 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:37:16.565695 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36908480-1558-4799-84d9-d1255e8f2c92 service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:37:16.572998 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc7de1c-6d3c-4a99-a762-ae1aec9867f0 service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:37:16.574727 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cfc7de1c-6d3c-4a99-a762-ae1aec9867f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:37:16.578883 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1412/2818] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:37:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:37:16.588424 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc7de1c-6d3c-4a99-a762-ae1aec9867f0 service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:37:16.594514 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f04bfa-d4c3-49aa-9363-44e608bb2d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:37:16.595491 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f04bfa-d4c3-49aa-9363-44e608bb2d9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:37:16.595776 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f04bfa-d4c3-49aa-9363-44e608bb2d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:37:16.596054 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f04bfa-d4c3-49aa-9363-44e608bb2d9a admin admin] PATH_INFO: `/v3/users/e28781af3a244fe5ae8eddc43bdddbe9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:37:16.597257 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f04bfa-d4c3-49aa-9363-44e608bb2d9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e28781af3a244fe5ae8eddc43bdddbe9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:37:16.599246 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f04bfa-d4c3-49aa-9363-44e608bb2d9a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:37:16.610980 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:37:16.647460 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d4f04bfa-d4c3-49aa-9363-44e608bb2d9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e28781af3a244fe5ae8eddc43bdddbe9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:37:16.648119 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d4f04bfa-d4c3-49aa-9363-44e608bb2d9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e28781af3a244fe5ae8eddc43bdddbe9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:37:16.648746 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d4f04bfa-d4c3-49aa-9363-44e608bb2d9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e28781af3a244fe5ae8eddc43bdddbe9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:37:16.656611 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d4f04bfa-d4c3-49aa-9363-44e608bb2d9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e28781af3a244fe5ae8eddc43bdddbe9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:37:16.665425 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d4f04bfa-d4c3-49aa-9363-44e608bb2d9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e28781af3a244fe5ae8eddc43bdddbe9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:37:16.684647 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1407/2819] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:37:16 2022] DELETE /identity/v3/users/e28781af3a244fe5ae8eddc43bdddbe9 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:37:16.858096 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36908480-1558-4799-84d9-d1255e8f2c92 service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:37:16.872521 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36908480-1558-4799-84d9-d1255e8f2c92 service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:37:16.918837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:37:16.934942 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f7c71f-cdad-4579-8768-c99e16aaaae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:37:16.935812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f7c71f-cdad-4579-8768-c99e16aaaae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:37:16.936103 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f7c71f-cdad-4579-8768-c99e16aaaae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:37:16.936437 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f7c71f-cdad-4579-8768-c99e16aaaae0 admin admin] PATH_INFO: `/v3/projects/a5d2c958ea024ee7be9e80a2af1392a7` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:37:16.937669 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f7c71f-cdad-4579-8768-c99e16aaaae0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5d2c958ea024ee7be9e80a2af1392a7)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:37:16.939722 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f7c71f-cdad-4579-8768-c99e16aaaae0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:37:16.991688 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-98f7c71f-cdad-4579-8768-c99e16aaaae0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5d2c958ea024ee7be9e80a2af1392a7', 'request_id': 'req-98f7c71f-cdad-4579-8768-c99e16aaaae0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:37:16.997782 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-98f7c71f-cdad-4579-8768-c99e16aaaae0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5d2c958ea024ee7be9e80a2af1392a7', 'request_id': 'req-98f7c71f-cdad-4579-8768-c99e16aaaae0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:37:17.005235 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-98f7c71f-cdad-4579-8768-c99e16aaaae0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5d2c958ea024ee7be9e80a2af1392a7', 'request_id': 'req-98f7c71f-cdad-4579-8768-c99e16aaaae0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:37:17.016050 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-98f7c71f-cdad-4579-8768-c99e16aaaae0 admin admin] The token cache is being invalidate because project a5d2c958ea024ee7be9e80a2af1392a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:37:17.016343 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-98f7c71f-cdad-4579-8768-c99e16aaaae0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:37:17.016970 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-98f7c71f-cdad-4579-8768-c99e16aaaae0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:37:17.018560 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1413/2820] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:37:16 2022] DELETE /identity/v3/projects/a5d2c958ea024ee7be9e80a2af1392a7 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:37:17.022479 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f04bfa-d4c3-49aa-9363-44e608bb2d9a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:37:17.023891 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d4f04bfa-d4c3-49aa-9363-44e608bb2d9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:37:17.037560 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f04bfa-d4c3-49aa-9363-44e608bb2d9a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:37:17.094700 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2593d64-3a3f-49dc-99d2-95caacb7a3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:37:17.095589 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2593d64-3a3f-49dc-99d2-95caacb7a3b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:37:17.095874 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2593d64-3a3f-49dc-99d2-95caacb7a3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:37:17.096163 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2593d64-3a3f-49dc-99d2-95caacb7a3b9 admin admin] PATH_INFO: `/v3/users/6e016293b78e470e901c3b0e81de65f6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:37:17.109529 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2593d64-3a3f-49dc-99d2-95caacb7a3b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e016293b78e470e901c3b0e81de65f6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:37:17.111544 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2593d64-3a3f-49dc-99d2-95caacb7a3b9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:37:17.157767 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b2593d64-3a3f-49dc-99d2-95caacb7a3b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e016293b78e470e901c3b0e81de65f6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:37:17.158433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b2593d64-3a3f-49dc-99d2-95caacb7a3b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e016293b78e470e901c3b0e81de65f6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:37:17.159023 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b2593d64-3a3f-49dc-99d2-95caacb7a3b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e016293b78e470e901c3b0e81de65f6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:37:17.165790 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b2593d64-3a3f-49dc-99d2-95caacb7a3b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e016293b78e470e901c3b0e81de65f6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:37:17.174131 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b2593d64-3a3f-49dc-99d2-95caacb7a3b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e016293b78e470e901c3b0e81de65f6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:37:17.185224 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:37:17.193326 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1408/2821] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:37:17 2022] DELETE /identity/v3/users/6e016293b78e470e901c3b0e81de65f6 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:37:17.379384 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f7c71f-cdad-4579-8768-c99e16aaaae0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:37:17.380868 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-98f7c71f-cdad-4579-8768-c99e16aaaae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:37:17.394264 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f7c71f-cdad-4579-8768-c99e16aaaae0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:37:17.452827 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c7118-2ad7-43fc-87e4-ec2e3d9fac37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:37:17.453732 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c7118-2ad7-43fc-87e4-ec2e3d9fac37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:37:17.454021 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c7118-2ad7-43fc-87e4-ec2e3d9fac37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:37:17.454300 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c7118-2ad7-43fc-87e4-ec2e3d9fac37 admin admin] PATH_INFO: `/v3/projects/114f7f29b73941ae83eef3fd58a6525f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:37:17.464860 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c7118-2ad7-43fc-87e4-ec2e3d9fac37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=114f7f29b73941ae83eef3fd58a6525f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:37:17.466630 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c7118-2ad7-43fc-87e4-ec2e3d9fac37 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:37:17.511464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c31c7118-2ad7-43fc-87e4-ec2e3d9fac37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '114f7f29b73941ae83eef3fd58a6525f', 'request_id': 'req-c31c7118-2ad7-43fc-87e4-ec2e3d9fac37'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:37:17.518684 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c31c7118-2ad7-43fc-87e4-ec2e3d9fac37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '114f7f29b73941ae83eef3fd58a6525f', 'request_id': 'req-c31c7118-2ad7-43fc-87e4-ec2e3d9fac37'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:37:17.525629 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c31c7118-2ad7-43fc-87e4-ec2e3d9fac37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '114f7f29b73941ae83eef3fd58a6525f', 'request_id': 'req-c31c7118-2ad7-43fc-87e4-ec2e3d9fac37'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:37:17.530504 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:37:17.537461 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c31c7118-2ad7-43fc-87e4-ec2e3d9fac37 admin admin] The token cache is being invalidate because project 114f7f29b73941ae83eef3fd58a6525f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:37:17.538197 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c31c7118-2ad7-43fc-87e4-ec2e3d9fac37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:37:17.539312 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c31c7118-2ad7-43fc-87e4-ec2e3d9fac37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:37:17.541175 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1414/2822] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:37:17 2022] DELETE /identity/v3/projects/114f7f29b73941ae83eef3fd58a6525f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:37:27.194564 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:37:27.541802 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:37:49.631388 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2593d64-3a3f-49dc-99d2-95caacb7a3b9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:37:49.633000 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b2593d64-3a3f-49dc-99d2-95caacb7a3b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:37:49.714863 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2593d64-3a3f-49dc-99d2-95caacb7a3b9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:37:49.766985 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa858cc-d5de-45d9-817c-34f1c5e7bd5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:37:49.767577 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa858cc-d5de-45d9-817c-34f1c5e7bd5c service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:37:49.767861 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa858cc-d5de-45d9-817c-34f1c5e7bd5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:37:49.768141 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa858cc-d5de-45d9-817c-34f1c5e7bd5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:37:49.769974 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-daa858cc-d5de-45d9-817c-34f1c5e7bd5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:37:49.837304 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa858cc-d5de-45d9-817c-34f1c5e7bd5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:37:49.838828 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa858cc-d5de-45d9-817c-34f1c5e7bd5c service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:37:49.848534 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1409/2823] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:37:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:37:53.123760 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c7118-2ad7-43fc-87e4-ec2e3d9fac37 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:37:53.125306 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c31c7118-2ad7-43fc-87e4-ec2e3d9fac37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:37:53.141424 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c7118-2ad7-43fc-87e4-ec2e3d9fac37 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:37:53.146495 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d83f9-3290-4f2b-a1d5-10d49be386a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:37:53.147648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d83f9-3290-4f2b-a1d5-10d49be386a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:37:53.147834 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d83f9-3290-4f2b-a1d5-10d49be386a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:37:53.148113 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d83f9-3290-4f2b-a1d5-10d49be386a7 admin admin] PATH_INFO: `/v3/users/805889a65fca41b38b334d7a7fdda4ac` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:37:53.149350 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d83f9-3290-4f2b-a1d5-10d49be386a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=805889a65fca41b38b334d7a7fdda4ac)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:37:53.151366 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d83f9-3290-4f2b-a1d5-10d49be386a7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:37:53.209888 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f67d83f9-3290-4f2b-a1d5-10d49be386a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '805889a65fca41b38b334d7a7fdda4ac'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:37:53.210889 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f67d83f9-3290-4f2b-a1d5-10d49be386a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '805889a65fca41b38b334d7a7fdda4ac'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:37:53.211266 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f67d83f9-3290-4f2b-a1d5-10d49be386a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '805889a65fca41b38b334d7a7fdda4ac'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:37:53.219841 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f67d83f9-3290-4f2b-a1d5-10d49be386a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '805889a65fca41b38b334d7a7fdda4ac'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:37:53.231417 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f67d83f9-3290-4f2b-a1d5-10d49be386a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '805889a65fca41b38b334d7a7fdda4ac'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:37:53.258893 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1415/2824] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:37:53 2022] DELETE /identity/v3/users/805889a65fca41b38b334d7a7fdda4ac => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:37:53.455525 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa858cc-d5de-45d9-817c-34f1c5e7bd5c service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:37:53.466761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa858cc-d5de-45d9-817c-34f1c5e7bd5c service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:37:53.517970 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:37:53.524909 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5fef2-d60a-4d7a-9379-84cf403b2ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:37:53.525884 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5fef2-d60a-4d7a-9379-84cf403b2ced admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:37:53.526168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5fef2-d60a-4d7a-9379-84cf403b2ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:37:53.526453 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5fef2-d60a-4d7a-9379-84cf403b2ced admin admin] PATH_INFO: `/v3/projects/0e53ca846fca4194a95fbdad22d10ed0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:37:53.527642 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c5fef2-d60a-4d7a-9379-84cf403b2ced admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e53ca846fca4194a95fbdad22d10ed0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:37:53.529568 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c5fef2-d60a-4d7a-9379-84cf403b2ced admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:37:53.576744 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f7c5fef2-d60a-4d7a-9379-84cf403b2ced admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e53ca846fca4194a95fbdad22d10ed0', 'request_id': 'req-f7c5fef2-d60a-4d7a-9379-84cf403b2ced'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:37:53.583955 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f7c5fef2-d60a-4d7a-9379-84cf403b2ced admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e53ca846fca4194a95fbdad22d10ed0', 'request_id': 'req-f7c5fef2-d60a-4d7a-9379-84cf403b2ced'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:37:53.589499 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f7c5fef2-d60a-4d7a-9379-84cf403b2ced admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e53ca846fca4194a95fbdad22d10ed0', 'request_id': 'req-f7c5fef2-d60a-4d7a-9379-84cf403b2ced'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:37:53.600481 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f7c5fef2-d60a-4d7a-9379-84cf403b2ced admin admin] The token cache is being invalidate because project 0e53ca846fca4194a95fbdad22d10ed0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:37:53.600773 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f7c5fef2-d60a-4d7a-9379-84cf403b2ced admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:37:53.601404 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f7c5fef2-d60a-4d7a-9379-84cf403b2ced admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:37:53.602998 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1410/2825] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:37:53 2022] DELETE /identity/v3/projects/0e53ca846fca4194a95fbdad22d10ed0 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:37:53.608859 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437eb9e6-1313-44e2-a9a4-38e6ed25bb36 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:37:53.609155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437eb9e6-1313-44e2-a9a4-38e6ed25bb36 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:37:53.609442 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437eb9e6-1313-44e2-a9a4-38e6ed25bb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:37:53.666559 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-437eb9e6-1313-44e2-a9a4-38e6ed25bb36 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:37:53.667716 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-437eb9e6-1313-44e2-a9a4-38e6ed25bb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:37:53.731245 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1416/2826] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:37:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:37:53.739361 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be72a9-2917-4cf4-afd2-a3660829a0d5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:37:53.739648 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be72a9-2917-4cf4-afd2-a3660829a0d5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:37:53.740181 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be72a9-2917-4cf4-afd2-a3660829a0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:37:53.759549 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-34be72a9-2917-4cf4-afd2-a3660829a0d5 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:37:53.760789 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-34be72a9-2917-4cf4-afd2-a3660829a0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:37:53.769312 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1411/2827] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:37:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:37:53.773964 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437eb9e6-1313-44e2-a9a4-38e6ed25bb36 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:37:53.783953 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437eb9e6-1313-44e2-a9a4-38e6ed25bb36 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:37:53.788436 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365293a-0628-442c-8829-6108d9f7d54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:37:53.789125 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365293a-0628-442c-8829-6108d9f7d54b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:37:53.789412 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365293a-0628-442c-8829-6108d9f7d54b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:37:53.789724 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365293a-0628-442c-8829-6108d9f7d54b admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:37:53.790463 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a365293a-0628-442c-8829-6108d9f7d54b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:37:53.790774 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a365293a-0628-442c-8829-6108d9f7d54b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:37:53.792533 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a365293a-0628-442c-8829-6108d9f7d54b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:37:53.805100 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1417/2828] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:37:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:37:53.809641 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be72a9-2917-4cf4-afd2-a3660829a0d5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:37:53.819803 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be72a9-2917-4cf4-afd2-a3660829a0d5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:37:53.824577 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0487a565-5032-4d5c-9c6b-1afaad138f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:37:53.825230 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487a565-5032-4d5c-9c6b-1afaad138f88 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:37:53.825511 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487a565-5032-4d5c-9c6b-1afaad138f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:37:53.825816 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487a565-5032-4d5c-9c6b-1afaad138f88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:37:53.826550 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0487a565-5032-4d5c-9c6b-1afaad138f88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:37:53.828478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0487a565-5032-4d5c-9c6b-1afaad138f88 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:37:53.866943 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1412/2829] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:37:53 2022] POST /identity/v3/projects => generated 363 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:37:53.871328 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365293a-0628-442c-8829-6108d9f7d54b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:37:53.882614 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365293a-0628-442c-8829-6108d9f7d54b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:37:53.908135 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:37:53.946919 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0750e-2179-422b-9486-f3fdb38ea69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:37:53.947503 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0750e-2179-422b-9486-f3fdb38ea69a admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:37:53.947797 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0750e-2179-422b-9486-f3fdb38ea69a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:37:53.948081 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0750e-2179-422b-9486-f3fdb38ea69a admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:37:53.948790 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b0750e-2179-422b-9486-f3fdb38ea69a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:37:53.950759 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b0750e-2179-422b-9486-f3fdb38ea69a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:37:53.973325 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1418/2830] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:37:53 2022] POST /identity/v3/users => generated 282 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:37:53.977486 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0487a565-5032-4d5c-9c6b-1afaad138f88 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:37:53.990971 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0487a565-5032-4d5c-9c6b-1afaad138f88 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:37:53.995761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d401aff8-d97e-4537-a4a6-a47ead1832a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:37:53.996423 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401aff8-d97e-4537-a4a6-a47ead1832a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:37:53.996702 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401aff8-d97e-4537-a4a6-a47ead1832a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:37:53.996976 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401aff8-d97e-4537-a4a6-a47ead1832a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:37:53.997785 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d401aff8-d97e-4537-a4a6-a47ead1832a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:37:54.000012 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d401aff8-d97e-4537-a4a6-a47ead1832a8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:37:54.010426 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1413/2831] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:37:54.014521 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0750e-2179-422b-9486-f3fdb38ea69a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:37:54.026839 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0750e-2179-422b-9486-f3fdb38ea69a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:37:54.033451 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec7918-f0c1-43ae-ae6e-fd524a8b8a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:37:54.034156 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec7918-f0c1-43ae-ae6e-fd524a8b8a6b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:37:54.034447 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec7918-f0c1-43ae-ae6e-fd524a8b8a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:37:54.034733 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec7918-f0c1-43ae-ae6e-fd524a8b8a6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:37:54.035479 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec7918-f0c1-43ae-ae6e-fd524a8b8a6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:37:54.037137 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec7918-f0c1-43ae-ae6e-fd524a8b8a6b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:37:54.048550 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1419/2832] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:37:54.052796 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d401aff8-d97e-4537-a4a6-a47ead1832a8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:37:54.063893 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d401aff8-d97e-4537-a4a6-a47ead1832a8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:37:54.068474 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf79bc7-bec3-4ae5-a383-f8115b1d80c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:37:54.069357 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf79bc7-bec3-4ae5-a383-f8115b1d80c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:37:54.069683 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf79bc7-bec3-4ae5-a383-f8115b1d80c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:37:54.069960 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf79bc7-bec3-4ae5-a383-f8115b1d80c5 admin admin] PATH_INFO: `/v3/projects/69dd866d8eb4475fb4e834cd87f265d6/users/2a2902d377764083b2215a9896a7fcd5/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:37:54.089600 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf79bc7-bec3-4ae5-a383-f8115b1d80c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69dd866d8eb4475fb4e834cd87f265d6, user_id=2a2902d377764083b2215a9896a7fcd5, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:37:54.093306 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf79bc7-bec3-4ae5-a383-f8115b1d80c5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:37:54.107544 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1414/2833] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:37:54 2022] PUT /identity/v3/projects/69dd866d8eb4475fb4e834cd87f265d6/users/2a2902d377764083b2215a9896a7fcd5/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:37:54.168898 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e852c8-041f-424d-b318-c877951bd3c9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:37:54.169553 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e852c8-041f-424d-b318-c877951bd3c9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:37:54.169553 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e852c8-041f-424d-b318-c877951bd3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:37:54.196142 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-17e852c8-041f-424d-b318-c877951bd3c9 None None] MFA Rules not processed for user `2a2902d377764083b2215a9896a7fcd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:37:54.197406 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-17e852c8-041f-424d-b318-c877951bd3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:37:54.262781 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1420/2834] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:37:54 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:37:54.267294 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf79bc7-bec3-4ae5-a383-f8115b1d80c5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:37:54.330395 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf79bc7-bec3-4ae5-a383-f8115b1d80c5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:37:54.337027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413ac18-5878-4160-9383-5936a4b28e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:37:54.337734 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413ac18-5878-4160-9383-5936a4b28e69 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:37:54.338128 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413ac18-5878-4160-9383-5936a4b28e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:37:54.338287 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413ac18-5878-4160-9383-5936a4b28e69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:37:54.339166 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413ac18-5878-4160-9383-5936a4b28e69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:37:54.341104 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413ac18-5878-4160-9383-5936a4b28e69 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:37:54.365450 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:37:54.378245 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1415/2835] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:37:54 2022] POST /identity/v3/projects => generated 365 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:37:54.379013 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e852c8-041f-424d-b318-c877951bd3c9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:37:54.391584 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e852c8-041f-424d-b318-c877951bd3c9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:37:54.452964 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d474b4c-bcb4-429c-91ff-1d3bc54b312a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:37:54.453691 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d474b4c-bcb4-429c-91ff-1d3bc54b312a admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:37:54.454032 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d474b4c-bcb4-429c-91ff-1d3bc54b312a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:37:54.454223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d474b4c-bcb4-429c-91ff-1d3bc54b312a admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:37:54.454943 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d474b4c-bcb4-429c-91ff-1d3bc54b312a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:37:54.456895 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d474b4c-bcb4-429c-91ff-1d3bc54b312a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:37:54.479539 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1421/2836] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:37:54 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:37:54.483824 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413ac18-5878-4160-9383-5936a4b28e69 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:37:54.494813 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413ac18-5878-4160-9383-5936a4b28e69 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:37:54.499378 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffccb0-dce6-4b44-be45-381c0378a5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:37:54.500037 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffccb0-dce6-4b44-be45-381c0378a5a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:37:54.500321 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffccb0-dce6-4b44-be45-381c0378a5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:37:54.500602 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffccb0-dce6-4b44-be45-381c0378a5a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:37:54.501334 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffccb0-dce6-4b44-be45-381c0378a5a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:37:54.502972 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffccb0-dce6-4b44-be45-381c0378a5a6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:37:54.514056 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1416/2837] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:37:54.518252 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d474b4c-bcb4-429c-91ff-1d3bc54b312a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:37:54.528231 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d474b4c-bcb4-429c-91ff-1d3bc54b312a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:37:54.534081 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6381b7-5ebb-498c-8094-d47967a8f749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:37:54.534955 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6381b7-5ebb-498c-8094-d47967a8f749 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:37:54.535239 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6381b7-5ebb-498c-8094-d47967a8f749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:37:54.535521 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6381b7-5ebb-498c-8094-d47967a8f749 admin admin] PATH_INFO: `/v3/projects/f862c5c001de4ce38f5d684ef47ffca3/users/51ceadc7e70044ab9a15df125a650db6/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:37:54.549983 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6381b7-5ebb-498c-8094-d47967a8f749 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f862c5c001de4ce38f5d684ef47ffca3, user_id=51ceadc7e70044ab9a15df125a650db6, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:37:54.552638 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6381b7-5ebb-498c-8094-d47967a8f749 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:37:54.565379 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1422/2838] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:37:54 2022] PUT /identity/v3/projects/f862c5c001de4ce38f5d684ef47ffca3/users/51ceadc7e70044ab9a15df125a650db6/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:37:54.569365 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffccb0-dce6-4b44-be45-381c0378a5a6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:37:54.580076 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffccb0-dce6-4b44-be45-381c0378a5a6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:37:54.634782 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586dbd86-ba15-41cf-aaee-cbdf653bcdd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:37:54.635446 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586dbd86-ba15-41cf-aaee-cbdf653bcdd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:37:54.635733 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586dbd86-ba15-41cf-aaee-cbdf653bcdd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:37:54.636037 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586dbd86-ba15-41cf-aaee-cbdf653bcdd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:37:54.636769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586dbd86-ba15-41cf-aaee-cbdf653bcdd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:37:54.638950 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586dbd86-ba15-41cf-aaee-cbdf653bcdd3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:37:54.648659 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1417/2839] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:37:54.652903 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6381b7-5ebb-498c-8094-d47967a8f749 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:37:54.659105 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:37:54.665284 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6381b7-5ebb-498c-8094-d47967a8f749 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:37:54.670293 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4813578-b9f1-4599-a88e-e47d59516955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:37:54.671176 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4813578-b9f1-4599-a88e-e47d59516955 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:37:54.671461 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4813578-b9f1-4599-a88e-e47d59516955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:37:54.671736 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4813578-b9f1-4599-a88e-e47d59516955 admin admin] PATH_INFO: `/v3/projects/f862c5c001de4ce38f5d684ef47ffca3/users/51ceadc7e70044ab9a15df125a650db6/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:37:54.673751 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4813578-b9f1-4599-a88e-e47d59516955 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f862c5c001de4ce38f5d684ef47ffca3, user_id=51ceadc7e70044ab9a15df125a650db6, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:37:54.676379 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4813578-b9f1-4599-a88e-e47d59516955 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:37:54.686745 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1423/2840] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:37:54 2022] PUT /identity/v3/projects/f862c5c001de4ce38f5d684ef47ffca3/users/51ceadc7e70044ab9a15df125a650db6/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:37:54.697101 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0ff81-4b91-4549-b795-b151e371a080 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:37:54.697390 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0ff81-4b91-4549-b795-b151e371a080 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:37:54.697762 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0ff81-4b91-4549-b795-b151e371a080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:37:54.724580 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a3d0ff81-4b91-4549-b795-b151e371a080 None None] MFA Rules not processed for user `51ceadc7e70044ab9a15df125a650db6`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:37:54.725808 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a3d0ff81-4b91-4549-b795-b151e371a080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:37:54.791804 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1418/2841] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:37:54 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:38:04.683045 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=4,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:38:04.788820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=69 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:38:10.773291 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4813578-b9f1-4599-a88e-e47d59516955 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:38:10.775042 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b4813578-b9f1-4599-a88e-e47d59516955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:38:10.847627 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4813578-b9f1-4599-a88e-e47d59516955 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:38:10.852970 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d267005-9fba-40a8-b05c-ef2cc17b2c7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:38:10.853666 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d267005-9fba-40a8-b05c-ef2cc17b2c7d service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:38:10.853952 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d267005-9fba-40a8-b05c-ef2cc17b2c7d service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:38:10.854233 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d267005-9fba-40a8-b05c-ef2cc17b2c7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:38:10.855989 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3d267005-9fba-40a8-b05c-ef2cc17b2c7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:38:10.935810 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d267005-9fba-40a8-b05c-ef2cc17b2c7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:38:10.937416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d267005-9fba-40a8-b05c-ef2cc17b2c7d service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:38:10.956907 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1424/2842] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:38:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:38:20.954177 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=69 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:38:53.760792 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0ff81-4b91-4549-b795-b151e371a080 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:38:53.763012 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a3d0ff81-4b91-4549-b795-b151e371a080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:38:53.849791 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0ff81-4b91-4549-b795-b151e371a080 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:38:53.855445 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4de1425-55dc-457e-900b-249b41fce581 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822d9d3521db4a9688f51c6313c7dd74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:38:53.856095 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de1425-55dc-457e-900b-249b41fce581 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:38:53.856380 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de1425-55dc-457e-900b-249b41fce581 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:38:53.856654 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de1425-55dc-457e-900b-249b41fce581 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:38:53.858463 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c4de1425-55dc-457e-900b-249b41fce581 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:38:53.870943 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4de1425-55dc-457e-900b-249b41fce581 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:38:53.872548 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4de1425-55dc-457e-900b-249b41fce581 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:38:53.882783 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1419/2843] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:38:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:38:54.878827 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d267005-9fba-40a8-b05c-ef2cc17b2c7d service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:38:54.951002 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d267005-9fba-40a8-b05c-ef2cc17b2c7d service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:38:54.956105 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce45113-ecfd-4920-8abb-5f544c2bb7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:38:54.956980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce45113-ecfd-4920-8abb-5f544c2bb7f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:38:54.957268 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce45113-ecfd-4920-8abb-5f544c2bb7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:38:54.957552 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce45113-ecfd-4920-8abb-5f544c2bb7f1 admin admin] PATH_INFO: `/v3/users/2a2902d377764083b2215a9896a7fcd5` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:38:54.958837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce45113-ecfd-4920-8abb-5f544c2bb7f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a2902d377764083b2215a9896a7fcd5)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:38:54.960802 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce45113-ecfd-4920-8abb-5f544c2bb7f1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:38:55.009111 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8ce45113-ecfd-4920-8abb-5f544c2bb7f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a2902d377764083b2215a9896a7fcd5'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:38:55.009828 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8ce45113-ecfd-4920-8abb-5f544c2bb7f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a2902d377764083b2215a9896a7fcd5'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:38:55.010441 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8ce45113-ecfd-4920-8abb-5f544c2bb7f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a2902d377764083b2215a9896a7fcd5'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:38:55.017824 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8ce45113-ecfd-4920-8abb-5f544c2bb7f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a2902d377764083b2215a9896a7fcd5'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:38:55.025186 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8ce45113-ecfd-4920-8abb-5f544c2bb7f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a2902d377764083b2215a9896a7fcd5'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:38:55.045063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1425/2844] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:38:54 2022] DELETE /identity/v3/users/2a2902d377764083b2215a9896a7fcd5 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:38:55.049972 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3145b40-3a0a-41a3-8d70-ca88097a5abe None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:38:55.050620 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3145b40-3a0a-41a3-8d70-ca88097a5abe None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:38:55.050620 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3145b40-3a0a-41a3-8d70-ca88097a5abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:38:55.069036 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f3145b40-3a0a-41a3-8d70-ca88097a5abe None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:38:55.070327 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f3145b40-3a0a-41a3-8d70-ca88097a5abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:38:55.140288 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1420/2845] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:38:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:38:55.153040 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce45113-ecfd-4920-8abb-5f544c2bb7f1 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:38:55.154631 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8ce45113-ecfd-4920-8abb-5f544c2bb7f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:38:55.219790 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce45113-ecfd-4920-8abb-5f544c2bb7f1 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:38:55.223827 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3dab0-a1bb-4bb8-9632-4d5c14072f66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:38:55.224475 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3dab0-a1bb-4bb8-9632-4d5c14072f66 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:38:55.224756 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3dab0-a1bb-4bb8-9632-4d5c14072f66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:38:55.225032 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3dab0-a1bb-4bb8-9632-4d5c14072f66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:38:55.235284 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c3dab0-a1bb-4bb8-9632-4d5c14072f66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:38:55.236747 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c3dab0-a1bb-4bb8-9632-4d5c14072f66 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:38:55.248166 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1426/2846] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:38:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:38:55.523288 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3145b40-3a0a-41a3-8d70-ca88097a5abe None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:38:55.524769 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f3145b40-3a0a-41a3-8d70-ca88097a5abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:38:55.536814 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3145b40-3a0a-41a3-8d70-ca88097a5abe None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:38:55.541675 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89ca46-d558-4d3c-a7d5-121decbb1d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:38:55.542558 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89ca46-d558-4d3c-a7d5-121decbb1d6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:38:55.542842 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89ca46-d558-4d3c-a7d5-121decbb1d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:38:55.543623 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89ca46-d558-4d3c-a7d5-121decbb1d6c admin admin] PATH_INFO: `/v3/projects/69dd866d8eb4475fb4e834cd87f265d6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:38:55.544840 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e89ca46-d558-4d3c-a7d5-121decbb1d6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=69dd866d8eb4475fb4e834cd87f265d6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:38:55.546817 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e89ca46-d558-4d3c-a7d5-121decbb1d6c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:38:55.605111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2e89ca46-d558-4d3c-a7d5-121decbb1d6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69dd866d8eb4475fb4e834cd87f265d6', 'request_id': 'req-2e89ca46-d558-4d3c-a7d5-121decbb1d6c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:38:55.614597 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2e89ca46-d558-4d3c-a7d5-121decbb1d6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69dd866d8eb4475fb4e834cd87f265d6', 'request_id': 'req-2e89ca46-d558-4d3c-a7d5-121decbb1d6c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:38:55.625515 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2e89ca46-d558-4d3c-a7d5-121decbb1d6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69dd866d8eb4475fb4e834cd87f265d6', 'request_id': 'req-2e89ca46-d558-4d3c-a7d5-121decbb1d6c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:38:55.638988 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2e89ca46-d558-4d3c-a7d5-121decbb1d6c admin admin] The token cache is being invalidate because project 69dd866d8eb4475fb4e834cd87f265d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:38:55.639281 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2e89ca46-d558-4d3c-a7d5-121decbb1d6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:38:55.639988 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2e89ca46-d558-4d3c-a7d5-121decbb1d6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:38:55.641544 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1421/2847] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:38:55 2022] DELETE /identity/v3/projects/69dd866d8eb4475fb4e834cd87f265d6 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:38:55.645813 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3dab0-a1bb-4bb8-9632-4d5c14072f66 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:38:55.658244 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3dab0-a1bb-4bb8-9632-4d5c14072f66 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:38:55.682490 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:38:55.725198 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794871e6-035c-4936-9a83-75a3c2e0cc98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:38:55.726109 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794871e6-035c-4936-9a83-75a3c2e0cc98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:38:55.726402 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794871e6-035c-4936-9a83-75a3c2e0cc98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:38:55.726684 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794871e6-035c-4936-9a83-75a3c2e0cc98 admin admin] PATH_INFO: `/v3/users/51ceadc7e70044ab9a15df125a650db6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:38:55.727910 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794871e6-035c-4936-9a83-75a3c2e0cc98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51ceadc7e70044ab9a15df125a650db6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:38:55.729886 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794871e6-035c-4936-9a83-75a3c2e0cc98 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:38:55.785322 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-794871e6-035c-4936-9a83-75a3c2e0cc98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51ceadc7e70044ab9a15df125a650db6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:38:55.786005 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-794871e6-035c-4936-9a83-75a3c2e0cc98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51ceadc7e70044ab9a15df125a650db6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:38:55.786629 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-794871e6-035c-4936-9a83-75a3c2e0cc98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51ceadc7e70044ab9a15df125a650db6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:38:55.794300 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-794871e6-035c-4936-9a83-75a3c2e0cc98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51ceadc7e70044ab9a15df125a650db6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:38:55.802128 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-794871e6-035c-4936-9a83-75a3c2e0cc98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51ceadc7e70044ab9a15df125a650db6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:38:55.821363 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1427/2848] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:38:55 2022] DELETE /identity/v3/users/51ceadc7e70044ab9a15df125a650db6 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 16:38:56.102961 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89ca46-d558-4d3c-a7d5-121decbb1d6c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:38:56.104505 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2e89ca46-d558-4d3c-a7d5-121decbb1d6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:38:56.113744 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:38:56.121472 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89ca46-d558-4d3c-a7d5-121decbb1d6c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:38:56.187103 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3cd132-b3cd-449b-bbe9-6669f556f0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:38:56.187908 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3cd132-b3cd-449b-bbe9-6669f556f0e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:38:56.188194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3cd132-b3cd-449b-bbe9-6669f556f0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:38:56.188473 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3cd132-b3cd-449b-bbe9-6669f556f0e5 admin admin] PATH_INFO: `/v3/projects/f862c5c001de4ce38f5d684ef47ffca3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:38:56.189703 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3cd132-b3cd-449b-bbe9-6669f556f0e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f862c5c001de4ce38f5d684ef47ffca3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:38:56.191643 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3cd132-b3cd-449b-bbe9-6669f556f0e5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:38:56.241186 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eb3cd132-b3cd-449b-bbe9-6669f556f0e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f862c5c001de4ce38f5d684ef47ffca3', 'request_id': 'req-eb3cd132-b3cd-449b-bbe9-6669f556f0e5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:38:56.249853 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eb3cd132-b3cd-449b-bbe9-6669f556f0e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f862c5c001de4ce38f5d684ef47ffca3', 'request_id': 'req-eb3cd132-b3cd-449b-bbe9-6669f556f0e5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:38:56.255687 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eb3cd132-b3cd-449b-bbe9-6669f556f0e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f862c5c001de4ce38f5d684ef47ffca3', 'request_id': 'req-eb3cd132-b3cd-449b-bbe9-6669f556f0e5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:38:56.267412 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eb3cd132-b3cd-449b-bbe9-6669f556f0e5 admin admin] The token cache is being invalidate because project f862c5c001de4ce38f5d684ef47ffca3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:38:56.267697 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eb3cd132-b3cd-449b-bbe9-6669f556f0e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:38:56.268336 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eb3cd132-b3cd-449b-bbe9-6669f556f0e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:38:56.269910 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1422/2849] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:38:56 2022] DELETE /identity/v3/projects/f862c5c001de4ce38f5d684ef47ffca3 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:38:56.274948 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a8d53-fd68-4b48-ad95-73718158c75a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:38:56.275462 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a8d53-fd68-4b48-ad95-73718158c75a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:38:56.275579 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a8d53-fd68-4b48-ad95-73718158c75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:38:56.294595 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-857a8d53-fd68-4b48-ad95-73718158c75a None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:38:56.295852 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-857a8d53-fd68-4b48-ad95-73718158c75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:38:56.356200 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1428/2850] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:38:56.366853 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321dc2cd-3299-405e-894c-9286af8281e5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:38:56.366853 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321dc2cd-3299-405e-894c-9286af8281e5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:38:56.366853 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321dc2cd-3299-405e-894c-9286af8281e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:38:56.386084 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-321dc2cd-3299-405e-894c-9286af8281e5 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:38:56.387961 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-321dc2cd-3299-405e-894c-9286af8281e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:38:56.396000 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1423/2851] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:38:56.400443 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a8d53-fd68-4b48-ad95-73718158c75a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:38:56.410421 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a8d53-fd68-4b48-ad95-73718158c75a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:38:56.415100 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2bc15-c40b-4afb-b99a-b168da7df68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:38:56.415905 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2bc15-c40b-4afb-b99a-b168da7df68d admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:38:56.416160 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2bc15-c40b-4afb-b99a-b168da7df68d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:38:56.416342 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2bc15-c40b-4afb-b99a-b168da7df68d admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:38:56.417082 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec2bc15-c40b-4afb-b99a-b168da7df68d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:38:56.417403 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec2bc15-c40b-4afb-b99a-b168da7df68d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:38:56.419053 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec2bc15-c40b-4afb-b99a-b168da7df68d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:38:56.431221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1429/2852] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:38:56.436290 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321dc2cd-3299-405e-894c-9286af8281e5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:38:56.446359 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321dc2cd-3299-405e-894c-9286af8281e5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:38:56.451765 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8c6cd-7e4e-4c87-9171-91d174b9174d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:38:56.452435 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8c6cd-7e4e-4c87-9171-91d174b9174d admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:38:56.452718 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8c6cd-7e4e-4c87-9171-91d174b9174d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:38:56.452995 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8c6cd-7e4e-4c87-9171-91d174b9174d admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:38:56.454100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c8c6cd-7e4e-4c87-9171-91d174b9174d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:38:56.456145 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c8c6cd-7e4e-4c87-9171-91d174b9174d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:38:56.483020 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1424/2853] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:38:56 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:38:56.487294 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2bc15-c40b-4afb-b99a-b168da7df68d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:38:56.498360 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2bc15-c40b-4afb-b99a-b168da7df68d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:38:56.524479 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:38:56.562144 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc0af2-b88e-41b6-8719-b64405e38798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:38:56.562836 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc0af2-b88e-41b6-8719-b64405e38798 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:38:56.563133 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc0af2-b88e-41b6-8719-b64405e38798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:38:56.563416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc0af2-b88e-41b6-8719-b64405e38798 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:38:56.564158 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc0af2-b88e-41b6-8719-b64405e38798 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:38:56.566241 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc0af2-b88e-41b6-8719-b64405e38798 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:38:56.588400 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1430/2854] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:38:56 2022] POST /identity/v3/users => generated 296 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:38:56.592726 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8c6cd-7e4e-4c87-9171-91d174b9174d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:38:56.604693 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8c6cd-7e4e-4c87-9171-91d174b9174d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:38:56.611581 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55ff52-52bc-475a-9ca4-64d31f9826dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:38:56.612640 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55ff52-52bc-475a-9ca4-64d31f9826dc admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:38:56.613108 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55ff52-52bc-475a-9ca4-64d31f9826dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:38:56.613562 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55ff52-52bc-475a-9ca4-64d31f9826dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:38:56.614845 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b55ff52-52bc-475a-9ca4-64d31f9826dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:38:56.617800 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b55ff52-52bc-475a-9ca4-64d31f9826dc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:38:56.629560 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1425/2855] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:38:56.633840 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc0af2-b88e-41b6-8719-b64405e38798 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:38:56.645106 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc0af2-b88e-41b6-8719-b64405e38798 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:38:56.649886 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba7850e-bcff-43ca-ba4c-53cc4ac76d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:38:56.650562 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba7850e-bcff-43ca-ba4c-53cc4ac76d97 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:38:56.650853 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba7850e-bcff-43ca-ba4c-53cc4ac76d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:38:56.651137 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba7850e-bcff-43ca-ba4c-53cc4ac76d97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:38:56.651888 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba7850e-bcff-43ca-ba4c-53cc4ac76d97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:38:56.653504 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba7850e-bcff-43ca-ba4c-53cc4ac76d97 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:38:56.666198 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1431/2856] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:38:56.672296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55ff52-52bc-475a-9ca4-64d31f9826dc admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:38:56.682997 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55ff52-52bc-475a-9ca4-64d31f9826dc admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:38:56.688522 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c1a39-c602-4142-9274-becbf31e2370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:38:56.689413 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c1a39-c602-4142-9274-becbf31e2370 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:38:56.689735 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c1a39-c602-4142-9274-becbf31e2370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:38:56.690027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c1a39-c602-4142-9274-becbf31e2370 admin admin] PATH_INFO: `/v3/projects/078234fa70204e63a612aa181a8c9097/users/0c208b6ae9fc4af28b1c278cbd22c9e7/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:38:56.706850 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c1a39-c602-4142-9274-becbf31e2370 admin admin] RBAC: Authorizing `identity:create_grant(project_id=078234fa70204e63a612aa181a8c9097, user_id=0c208b6ae9fc4af28b1c278cbd22c9e7, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:38:56.709503 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c1a39-c602-4142-9274-becbf31e2370 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:38:56.722211 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1426/2857] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:38:56 2022] PUT /identity/v3/projects/078234fa70204e63a612aa181a8c9097/users/0c208b6ae9fc4af28b1c278cbd22c9e7/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:38:56.732400 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e514b1-e9ae-4621-a9b5-4787350108e7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:38:56.732728 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e514b1-e9ae-4621-a9b5-4787350108e7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:38:56.733015 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e514b1-e9ae-4621-a9b5-4787350108e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:38:56.769356 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-87e514b1-e9ae-4621-a9b5-4787350108e7 None None] MFA Rules not processed for user `0c208b6ae9fc4af28b1c278cbd22c9e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:38:56.770630 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-87e514b1-e9ae-4621-a9b5-4787350108e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:38:56.833990 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1432/2858] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:38:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:38:56.838490 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c1a39-c602-4142-9274-becbf31e2370 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:38:56.869877 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:38:56.902154 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c1a39-c602-4142-9274-becbf31e2370 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:38:56.906902 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa14669-84b3-41e3-a480-c6917d3d6c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:38:56.907577 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa14669-84b3-41e3-a480-c6917d3d6c6c admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:38:56.907865 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa14669-84b3-41e3-a480-c6917d3d6c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:38:56.908146 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa14669-84b3-41e3-a480-c6917d3d6c6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:38:56.908875 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa14669-84b3-41e3-a480-c6917d3d6c6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:38:56.910882 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa14669-84b3-41e3-a480-c6917d3d6c6c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:38:56.941515 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1427/2859] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:38:56 2022] POST /identity/v3/projects => generated 391 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:38:56.943370 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e514b1-e9ae-4621-a9b5-4787350108e7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:38:56.954354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e514b1-e9ae-4621-a9b5-4787350108e7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:38:57.012564 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6896f-305b-4ee8-8117-56d82a456620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:38:57.013155 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6896f-305b-4ee8-8117-56d82a456620 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:38:57.013447 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6896f-305b-4ee8-8117-56d82a456620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:38:57.013764 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6896f-305b-4ee8-8117-56d82a456620 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:38:57.014468 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6896f-305b-4ee8-8117-56d82a456620 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:38:57.016401 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6896f-305b-4ee8-8117-56d82a456620 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:38:57.036354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1433/2860] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:38:56 2022] POST /identity/v3/users => generated 302 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:38:57.040477 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa14669-84b3-41e3-a480-c6917d3d6c6c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:38:57.053024 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa14669-84b3-41e3-a480-c6917d3d6c6c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:38:57.057869 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45289eb8-e82d-4a47-ad18-89b3116e37c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:38:57.058560 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45289eb8-e82d-4a47-ad18-89b3116e37c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:38:57.059476 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45289eb8-e82d-4a47-ad18-89b3116e37c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:38:57.059761 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45289eb8-e82d-4a47-ad18-89b3116e37c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:38:57.060497 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45289eb8-e82d-4a47-ad18-89b3116e37c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:38:57.062175 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45289eb8-e82d-4a47-ad18-89b3116e37c1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:38:57.073357 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1428/2861] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:38:57.077985 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6896f-305b-4ee8-8117-56d82a456620 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:38:57.088211 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6896f-305b-4ee8-8117-56d82a456620 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:38:57.093110 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f03ed-f999-4100-a004-2466717320d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:38:57.094135 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f03ed-f999-4100-a004-2466717320d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:38:57.094312 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f03ed-f999-4100-a004-2466717320d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:38:57.094585 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f03ed-f999-4100-a004-2466717320d1 admin admin] PATH_INFO: `/v3/projects/22ee6adebec94d2fa14c4b6cc894b294/users/e1913020e979430e81d07b6fe6b8b5e3/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:38:57.114101 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2f03ed-f999-4100-a004-2466717320d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22ee6adebec94d2fa14c4b6cc894b294, user_id=e1913020e979430e81d07b6fe6b8b5e3, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:38:57.117073 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2f03ed-f999-4100-a004-2466717320d1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:38:57.139770 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1434/2862] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:38:57 2022] PUT /identity/v3/projects/22ee6adebec94d2fa14c4b6cc894b294/users/e1913020e979430e81d07b6fe6b8b5e3/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:38:57.143885 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45289eb8-e82d-4a47-ad18-89b3116e37c1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:38:57.155739 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45289eb8-e82d-4a47-ad18-89b3116e37c1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:38:57.220388 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6442ba-4c52-4169-9b62-6beb5e433766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:38:57.221100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6442ba-4c52-4169-9b62-6beb5e433766 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:38:57.221417 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6442ba-4c52-4169-9b62-6beb5e433766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:38:57.221730 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6442ba-4c52-4169-9b62-6beb5e433766 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:38:57.222629 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6442ba-4c52-4169-9b62-6beb5e433766 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:38:57.224288 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6442ba-4c52-4169-9b62-6beb5e433766 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:38:57.235433 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1429/2863] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:38:57.239815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f03ed-f999-4100-a004-2466717320d1 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:38:57.245927 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:38:57.253223 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f03ed-f999-4100-a004-2466717320d1 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:38:57.258221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40407c8c-f9fd-4beb-8273-5493e2d297cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:38:57.258898 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40407c8c-f9fd-4beb-8273-5493e2d297cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:38:57.259178 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40407c8c-f9fd-4beb-8273-5493e2d297cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:38:57.259456 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40407c8c-f9fd-4beb-8273-5493e2d297cb admin admin] PATH_INFO: `/v3/projects/22ee6adebec94d2fa14c4b6cc894b294/users/e1913020e979430e81d07b6fe6b8b5e3/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:38:57.261783 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40407c8c-f9fd-4beb-8273-5493e2d297cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=22ee6adebec94d2fa14c4b6cc894b294, user_id=e1913020e979430e81d07b6fe6b8b5e3, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:38:57.264497 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40407c8c-f9fd-4beb-8273-5493e2d297cb admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:38:57.275179 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1435/2864] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:38:57 2022] PUT /identity/v3/projects/22ee6adebec94d2fa14c4b6cc894b294/users/e1913020e979430e81d07b6fe6b8b5e3/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:38:57.286664 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4c89d-a346-4571-ba8c-7f2f9c336725 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:38:57.286962 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4c89d-a346-4571-ba8c-7f2f9c336725 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:38:57.287246 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4c89d-a346-4571-ba8c-7f2f9c336725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:38:57.315842 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-38d4c89d-a346-4571-ba8c-7f2f9c336725 None None] MFA Rules not processed for user `e1913020e979430e81d07b6fe6b8b5e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:38:57.317069 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-38d4c89d-a346-4571-ba8c-7f2f9c336725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:38:57.389268 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1430/2865] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:38:57 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:39:07.271333 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=4,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:39:07.386046 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=88,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:39:49.725101 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40407c8c-f9fd-4beb-8273-5493e2d297cb admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:39:49.726509 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-40407c8c-f9fd-4beb-8273-5493e2d297cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:39:49.809748 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40407c8c-f9fd-4beb-8273-5493e2d297cb admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:39:49.814144 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5541c8-6d7b-40d5-a7af-57b63d0df5a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:39:49.814797 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5541c8-6d7b-40d5-a7af-57b63d0df5a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:39:49.815088 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5541c8-6d7b-40d5-a7af-57b63d0df5a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:39:49.815371 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5541c8-6d7b-40d5-a7af-57b63d0df5a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:39:49.817193 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ed5541c8-6d7b-40d5-a7af-57b63d0df5a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:39:49.877618 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5541c8-6d7b-40d5-a7af-57b63d0df5a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:39:49.879095 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5541c8-6d7b-40d5-a7af-57b63d0df5a1 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:39:49.888893 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1436/2866] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:39:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:39:49.915654 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4c89d-a346-4571-ba8c-7f2f9c336725 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:39:49.925786 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4c89d-a346-4571-ba8c-7f2f9c336725 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:39:49.929296 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669db0f6-5332-437a-bee9-ddcead684726 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:39:49.930000 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669db0f6-5332-437a-bee9-ddcead684726 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:39:49.930286 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669db0f6-5332-437a-bee9-ddcead684726 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:39:49.930567 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669db0f6-5332-437a-bee9-ddcead684726 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:39:49.932329 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-669db0f6-5332-437a-bee9-ddcead684726 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:39:49.945820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-669db0f6-5332-437a-bee9-ddcead684726 service cinder] Could not find project: f862c5c001de4ce38f5d684ef47ffca3.: keystone.exception.ProjectNotFound: Could not find project: f862c5c001de4ce38f5d684ef47ffca3. Jul 21 16:39:49.945820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 16:39:49.945820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 16:39:49.945820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 16:39:49.945820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 16:39:49.945820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 16:39:49.945820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 16:39:49.945820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 16:39:49.945820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 16:39:49.945820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 16:39:49.945820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 16:39:49.945820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 16:39:49.945820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Jul 21 16:39:49.945820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Jul 21 16:39:49.945820 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Jul 21 16:39:49.947619 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Jul 21 16:39:49.947619 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Jul 21 16:39:49.947619 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Jul 21 16:39:49.947619 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 16:39:49.947619 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 16:39:49.947619 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Jul 21 16:39:49.947619 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Jul 21 16:39:49.947619 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 16:39:49.947619 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 16:39:49.947619 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 16:39:49.947619 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 16:39:49.947619 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 16:39:49.947619 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 16:39:49.947619 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 16:39:49.947619 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 16:39:49.947619 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 16:39:49.947619 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 16:39:49.949279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 16:39:49.949279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 16:39:49.949279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 16:39:49.949279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 16:39:49.949279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Jul 21 16:39:49.949279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Jul 21 16:39:49.949279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Jul 21 16:39:49.949279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._validate_token_resources() Jul 21 16:39:49.949279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Jul 21 16:39:49.949279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Jul 21 16:39:49.949279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Jul 21 16:39:49.949279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Jul 21 16:39:49.949279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 16:39:49.949279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 16:39:49.949279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 16:39:49.949279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 16:39:49.949279 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 16:39:49.950829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 16:39:49.950829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 16:39:49.950829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 21 16:39:49.950829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 16:39:49.950829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 21 16:39:49.950829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 16:39:49.950829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 16:39:49.950829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 16:39:49.950829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 21 16:39:49.950829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 16:39:49.950829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 21 16:39:49.950829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 16:39:49.950829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 16:39:49.950829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 16:39:49.950829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 16:39:49.950829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 16:39:49.950829 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 16:39:49.952379 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: f862c5c001de4ce38f5d684ef47ffca3. Jul 21 16:39:49.952379 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 21 16:39:49.952379 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1431/2867] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:39:49 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 32 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Jul 21 16:39:50.002933 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5541c8-6d7b-40d5-a7af-57b63d0df5a1 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:39:50.013062 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5541c8-6d7b-40d5-a7af-57b63d0df5a1 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:39:50.015371 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a10d50c-179d-4bd9-aa0f-94e1aa7a3ae4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:39:50.016007 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a10d50c-179d-4bd9-aa0f-94e1aa7a3ae4 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:39:50.016296 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a10d50c-179d-4bd9-aa0f-94e1aa7a3ae4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:39:50.016577 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a10d50c-179d-4bd9-aa0f-94e1aa7a3ae4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:39:50.025708 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a10d50c-179d-4bd9-aa0f-94e1aa7a3ae4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:39:50.027147 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a10d50c-179d-4bd9-aa0f-94e1aa7a3ae4 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:39:50.036546 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1437/2868] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:39:50 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:39:59.935913 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:40:00.032357 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=63 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:41:09.155358 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669db0f6-5332-437a-bee9-ddcead684726 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:41:09.156800 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-669db0f6-5332-437a-bee9-ddcead684726 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:41:09.214212 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669db0f6-5332-437a-bee9-ddcead684726 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:41:09.217882 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94553146-a9f3-4929-a638-c9b4d13b2321 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:41:09.218567 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94553146-a9f3-4929-a638-c9b4d13b2321 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:09.218812 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94553146-a9f3-4929-a638-c9b4d13b2321 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:09.219091 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94553146-a9f3-4929-a638-c9b4d13b2321 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:09.220855 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-94553146-a9f3-4929-a638-c9b4d13b2321 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:41:09.292257 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94553146-a9f3-4929-a638-c9b4d13b2321 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:41:09.293738 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94553146-a9f3-4929-a638-c9b4d13b2321 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:41:09.305283 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1432/2869] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:41:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:41:10.718066 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a10d50c-179d-4bd9-aa0f-94e1aa7a3ae4 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:41:10.719567 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0a10d50c-179d-4bd9-aa0f-94e1aa7a3ae4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:41:10.730799 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a10d50c-179d-4bd9-aa0f-94e1aa7a3ae4 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:41:10.734436 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df5693e-6744-4fcd-8721-15ab45d9b6a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:41:10.735068 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5693e-6744-4fcd-8721-15ab45d9b6a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:10.735356 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5693e-6744-4fcd-8721-15ab45d9b6a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:10.735640 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5693e-6744-4fcd-8721-15ab45d9b6a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:10.737451 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8df5693e-6744-4fcd-8721-15ab45d9b6a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:41:10.751954 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-8df5693e-6744-4fcd-8721-15ab45d9b6a6 service cinder] Could not find project: a5d2c958ea024ee7be9e80a2af1392a7.: keystone.exception.ProjectNotFound: Could not find project: a5d2c958ea024ee7be9e80a2af1392a7. Jul 21 16:41:10.751954 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 16:41:10.751954 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 16:41:10.751954 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 16:41:10.751954 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 16:41:10.751954 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 16:41:10.751954 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 16:41:10.751954 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 16:41:10.751954 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 16:41:10.751954 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 16:41:10.751954 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 16:41:10.751954 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 16:41:10.751954 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Jul 21 16:41:10.751954 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Jul 21 16:41:10.751954 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Jul 21 16:41:10.753289 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Jul 21 16:41:10.753289 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Jul 21 16:41:10.753289 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Jul 21 16:41:10.753289 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 16:41:10.753289 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 16:41:10.753289 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Jul 21 16:41:10.753289 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Jul 21 16:41:10.753289 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 16:41:10.753289 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 16:41:10.753289 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 16:41:10.753289 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 16:41:10.753289 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 16:41:10.753289 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 16:41:10.753289 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 16:41:10.753289 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 16:41:10.753289 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 16:41:10.753289 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 16:41:10.755150 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 16:41:10.755150 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 16:41:10.755150 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 16:41:10.755150 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 16:41:10.755150 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Jul 21 16:41:10.755150 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Jul 21 16:41:10.755150 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Jul 21 16:41:10.755150 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._validate_token_resources() Jul 21 16:41:10.755150 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Jul 21 16:41:10.755150 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Jul 21 16:41:10.755150 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Jul 21 16:41:10.755150 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Jul 21 16:41:10.755150 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 16:41:10.755150 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 16:41:10.755150 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 16:41:10.755150 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 16:41:10.755150 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 16:41:10.756354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 16:41:10.756354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 16:41:10.756354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 21 16:41:10.756354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 16:41:10.756354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 21 16:41:10.756354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 16:41:10.756354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 16:41:10.756354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 16:41:10.756354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 21 16:41:10.756354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 16:41:10.756354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 21 16:41:10.756354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 16:41:10.756354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 16:41:10.756354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 16:41:10.756354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 16:41:10.756354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 16:41:10.756354 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 16:41:10.757646 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: a5d2c958ea024ee7be9e80a2af1392a7. Jul 21 16:41:10.757646 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 21 16:41:10.757646 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1438/2870] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:41:10 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 36 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Jul 21 16:41:10.786328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94553146-a9f3-4929-a638-c9b4d13b2321 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:41:10.795860 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94553146-a9f3-4929-a638-c9b4d13b2321 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:41:10.799561 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493db946-09af-4126-8ff4-1f2a961cf47f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:41:10.800399 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493db946-09af-4126-8ff4-1f2a961cf47f service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:10.800757 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493db946-09af-4126-8ff4-1f2a961cf47f service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:10.801683 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493db946-09af-4126-8ff4-1f2a961cf47f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:10.864759 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493db946-09af-4126-8ff4-1f2a961cf47f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:41:10.866489 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493db946-09af-4126-8ff4-1f2a961cf47f service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:41:10.875302 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1433/2871] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:41:10 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:41:11.308282 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df5693e-6744-4fcd-8721-15ab45d9b6a6 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:41:11.377074 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df5693e-6744-4fcd-8721-15ab45d9b6a6 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:41:11.382377 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99921516-73a9-45ed-b03c-45913db6b28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:41:11.383250 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99921516-73a9-45ed-b03c-45913db6b28a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:11.383536 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99921516-73a9-45ed-b03c-45913db6b28a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:11.383818 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99921516-73a9-45ed-b03c-45913db6b28a admin admin] PATH_INFO: `/v3/users/0c208b6ae9fc4af28b1c278cbd22c9e7` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:11.385018 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99921516-73a9-45ed-b03c-45913db6b28a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c208b6ae9fc4af28b1c278cbd22c9e7)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:41:11.386981 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99921516-73a9-45ed-b03c-45913db6b28a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:41:11.433304 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-99921516-73a9-45ed-b03c-45913db6b28a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c208b6ae9fc4af28b1c278cbd22c9e7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:41:11.433976 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-99921516-73a9-45ed-b03c-45913db6b28a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c208b6ae9fc4af28b1c278cbd22c9e7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:41:11.434604 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-99921516-73a9-45ed-b03c-45913db6b28a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c208b6ae9fc4af28b1c278cbd22c9e7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:41:11.441509 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-99921516-73a9-45ed-b03c-45913db6b28a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c208b6ae9fc4af28b1c278cbd22c9e7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:41:11.449449 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-99921516-73a9-45ed-b03c-45913db6b28a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c208b6ae9fc4af28b1c278cbd22c9e7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:41:11.468070 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1439/2872] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:41:11 2022] DELETE /identity/v3/users/0c208b6ae9fc4af28b1c278cbd22c9e7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:41:11.472436 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c87764-a8ee-4412-b77d-a4cdbc19bf63 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:11.472728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c87764-a8ee-4412-b77d-a4cdbc19bf63 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:11.473021 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c87764-a8ee-4412-b77d-a4cdbc19bf63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:11.491015 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d7c87764-a8ee-4412-b77d-a4cdbc19bf63 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:41:11.492212 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d7c87764-a8ee-4412-b77d-a4cdbc19bf63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:41:11.527149 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:41:11.553503 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1434/2873] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:41:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:41:11.564604 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99921516-73a9-45ed-b03c-45913db6b28a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:41:11.566104 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-99921516-73a9-45ed-b03c-45913db6b28a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:41:11.632424 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99921516-73a9-45ed-b03c-45913db6b28a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:41:11.636122 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7b0e8-e1bb-4c74-ae74-626d9daef96a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:41:11.636780 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7b0e8-e1bb-4c74-ae74-626d9daef96a service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:11.637067 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7b0e8-e1bb-4c74-ae74-626d9daef96a service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:11.637379 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7b0e8-e1bb-4c74-ae74-626d9daef96a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:11.647517 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b7b0e8-e1bb-4c74-ae74-626d9daef96a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:41:11.648963 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b7b0e8-e1bb-4c74-ae74-626d9daef96a service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:41:11.658698 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1440/2874] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:41:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:41:11.954192 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c87764-a8ee-4412-b77d-a4cdbc19bf63 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:41:11.956905 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d7c87764-a8ee-4412-b77d-a4cdbc19bf63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:41:11.970583 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c87764-a8ee-4412-b77d-a4cdbc19bf63 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:41:11.977810 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7641850-92f9-4a34-8ee0-a953daaf530e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:41:11.979298 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7641850-92f9-4a34-8ee0-a953daaf530e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:11.979783 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7641850-92f9-4a34-8ee0-a953daaf530e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:11.980268 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7641850-92f9-4a34-8ee0-a953daaf530e admin admin] PATH_INFO: `/v3/projects/078234fa70204e63a612aa181a8c9097` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:11.982843 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7641850-92f9-4a34-8ee0-a953daaf530e admin admin] RBAC: Authorizing `identity:delete_project(project_id=078234fa70204e63a612aa181a8c9097)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:41:11.986872 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7641850-92f9-4a34-8ee0-a953daaf530e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:41:12.042656 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b7641850-92f9-4a34-8ee0-a953daaf530e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '078234fa70204e63a612aa181a8c9097', 'request_id': 'req-b7641850-92f9-4a34-8ee0-a953daaf530e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:41:12.056155 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b7641850-92f9-4a34-8ee0-a953daaf530e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '078234fa70204e63a612aa181a8c9097', 'request_id': 'req-b7641850-92f9-4a34-8ee0-a953daaf530e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:41:12.061596 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b7641850-92f9-4a34-8ee0-a953daaf530e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '078234fa70204e63a612aa181a8c9097', 'request_id': 'req-b7641850-92f9-4a34-8ee0-a953daaf530e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:41:12.073479 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b7641850-92f9-4a34-8ee0-a953daaf530e admin admin] The token cache is being invalidate because project 078234fa70204e63a612aa181a8c9097 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:41:12.073802 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b7641850-92f9-4a34-8ee0-a953daaf530e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:41:12.074521 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b7641850-92f9-4a34-8ee0-a953daaf530e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:41:12.076079 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1435/2875] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:41:11 2022] DELETE /identity/v3/projects/078234fa70204e63a612aa181a8c9097 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:41:12.080871 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7b0e8-e1bb-4c74-ae74-626d9daef96a service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:41:12.092990 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7b0e8-e1bb-4c74-ae74-626d9daef96a service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:41:12.102500 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:41:12.151695 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c8b58-c907-4ef7-ad54-35aaf1bd2646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:41:12.152576 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c8b58-c907-4ef7-ad54-35aaf1bd2646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:12.152867 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c8b58-c907-4ef7-ad54-35aaf1bd2646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:12.153151 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c8b58-c907-4ef7-ad54-35aaf1bd2646 admin admin] PATH_INFO: `/v3/users/e1913020e979430e81d07b6fe6b8b5e3` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:12.154374 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2c8b58-c907-4ef7-ad54-35aaf1bd2646 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1913020e979430e81d07b6fe6b8b5e3)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:41:12.156302 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2c8b58-c907-4ef7-ad54-35aaf1bd2646 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:41:12.202915 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bf2c8b58-c907-4ef7-ad54-35aaf1bd2646 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1913020e979430e81d07b6fe6b8b5e3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:41:12.203579 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bf2c8b58-c907-4ef7-ad54-35aaf1bd2646 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1913020e979430e81d07b6fe6b8b5e3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:41:12.204180 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bf2c8b58-c907-4ef7-ad54-35aaf1bd2646 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1913020e979430e81d07b6fe6b8b5e3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:41:12.211794 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bf2c8b58-c907-4ef7-ad54-35aaf1bd2646 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1913020e979430e81d07b6fe6b8b5e3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:41:12.218872 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bf2c8b58-c907-4ef7-ad54-35aaf1bd2646 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1913020e979430e81d07b6fe6b8b5e3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:41:12.236760 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1441/2876] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:41:12 2022] DELETE /identity/v3/users/e1913020e979430e81d07b6fe6b8b5e3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:41:12.512828 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7641850-92f9-4a34-8ee0-a953daaf530e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:41:12.514324 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b7641850-92f9-4a34-8ee0-a953daaf530e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:41:12.528358 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7641850-92f9-4a34-8ee0-a953daaf530e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:41:12.587055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a878d03f-8d03-4503-bdd0-5c5ff9238928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:41:12.587839 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878d03f-8d03-4503-bdd0-5c5ff9238928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:12.588116 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878d03f-8d03-4503-bdd0-5c5ff9238928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:12.588384 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878d03f-8d03-4503-bdd0-5c5ff9238928 admin admin] PATH_INFO: `/v3/projects/22ee6adebec94d2fa14c4b6cc894b294` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:12.589550 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a878d03f-8d03-4503-bdd0-5c5ff9238928 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22ee6adebec94d2fa14c4b6cc894b294)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:41:12.591490 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a878d03f-8d03-4503-bdd0-5c5ff9238928 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:41:12.639043 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a878d03f-8d03-4503-bdd0-5c5ff9238928 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22ee6adebec94d2fa14c4b6cc894b294', 'request_id': 'req-a878d03f-8d03-4503-bdd0-5c5ff9238928'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:41:12.647151 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a878d03f-8d03-4503-bdd0-5c5ff9238928 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22ee6adebec94d2fa14c4b6cc894b294', 'request_id': 'req-a878d03f-8d03-4503-bdd0-5c5ff9238928'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:41:12.652764 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a878d03f-8d03-4503-bdd0-5c5ff9238928 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22ee6adebec94d2fa14c4b6cc894b294', 'request_id': 'req-a878d03f-8d03-4503-bdd0-5c5ff9238928'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:41:12.653853 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:41:12.665879 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a878d03f-8d03-4503-bdd0-5c5ff9238928 admin admin] The token cache is being invalidate because project 22ee6adebec94d2fa14c4b6cc894b294 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:41:12.666163 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a878d03f-8d03-4503-bdd0-5c5ff9238928 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:41:12.666789 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a878d03f-8d03-4503-bdd0-5c5ff9238928 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:41:12.668403 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1436/2877] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:41:12 2022] DELETE /identity/v3/projects/22ee6adebec94d2fa14c4b6cc894b294 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:41:12.677880 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980f483-bcdd-4938-86d4-206a4e37807f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:12.678178 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980f483-bcdd-4938-86d4-206a4e37807f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:12.678462 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980f483-bcdd-4938-86d4-206a4e37807f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:12.697310 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9980f483-bcdd-4938-86d4-206a4e37807f None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:41:12.698556 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9980f483-bcdd-4938-86d4-206a4e37807f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:41:12.758567 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1442/2878] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:41:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:41:12.767328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefd38c-c134-4dcf-98a4-0a3519ad0d3a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:12.767620 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefd38c-c134-4dcf-98a4-0a3519ad0d3a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:12.767903 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefd38c-c134-4dcf-98a4-0a3519ad0d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:12.786664 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-deefd38c-c134-4dcf-98a4-0a3519ad0d3a None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:41:12.787889 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-deefd38c-c134-4dcf-98a4-0a3519ad0d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:41:12.795623 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1437/2879] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:41:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:41:12.799972 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980f483-bcdd-4938-86d4-206a4e37807f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:41:12.810063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980f483-bcdd-4938-86d4-206a4e37807f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:41:12.814612 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506086a9-d751-4b28-8934-3b1e14598d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:41:12.815266 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506086a9-d751-4b28-8934-3b1e14598d71 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:12.815561 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506086a9-d751-4b28-8934-3b1e14598d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:12.815847 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506086a9-d751-4b28-8934-3b1e14598d71 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:12.816619 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506086a9-d751-4b28-8934-3b1e14598d71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:41:12.817414 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506086a9-d751-4b28-8934-3b1e14598d71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:41:12.819053 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506086a9-d751-4b28-8934-3b1e14598d71 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:41:12.830753 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1443/2880] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:41:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:41:12.835675 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefd38c-c134-4dcf-98a4-0a3519ad0d3a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:41:12.845228 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefd38c-c134-4dcf-98a4-0a3519ad0d3a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:41:12.850014 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58cc0b-7544-4f0d-a9e5-bce83d4733b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:41:12.850648 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58cc0b-7544-4f0d-a9e5-bce83d4733b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:12.850928 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58cc0b-7544-4f0d-a9e5-bce83d4733b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:12.851200 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58cc0b-7544-4f0d-a9e5-bce83d4733b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:12.851877 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58cc0b-7544-4f0d-a9e5-bce83d4733b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:41:12.853792 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58cc0b-7544-4f0d-a9e5-bce83d4733b7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:41:12.880446 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1438/2881] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:41:12 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:41:12.884560 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506086a9-d751-4b28-8934-3b1e14598d71 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:41:12.894815 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506086a9-d751-4b28-8934-3b1e14598d71 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:41:12.913857 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:41:12.954288 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271b5cb-d185-4b83-a56c-68c1e54e4e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:41:12.954965 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271b5cb-d185-4b83-a56c-68c1e54e4e57 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:12.955249 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271b5cb-d185-4b83-a56c-68c1e54e4e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:12.955529 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271b5cb-d185-4b83-a56c-68c1e54e4e57 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:12.956215 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271b5cb-d185-4b83-a56c-68c1e54e4e57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:41:12.958646 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271b5cb-d185-4b83-a56c-68c1e54e4e57 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:41:12.978809 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1444/2882] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:41:12 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 16:41:12.983328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58cc0b-7544-4f0d-a9e5-bce83d4733b7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:41:12.994640 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58cc0b-7544-4f0d-a9e5-bce83d4733b7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:41:12.999303 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36173494-1a08-436d-9897-f8544cc4b780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:41:12.999960 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36173494-1a08-436d-9897-f8544cc4b780 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:13.000247 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36173494-1a08-436d-9897-f8544cc4b780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:13.000527 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36173494-1a08-436d-9897-f8544cc4b780 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:13.001259 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36173494-1a08-436d-9897-f8544cc4b780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:41:13.002910 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36173494-1a08-436d-9897-f8544cc4b780 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:41:13.013953 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1439/2883] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:41:13.017827 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271b5cb-d185-4b83-a56c-68c1e54e4e57 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:41:13.027732 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271b5cb-d185-4b83-a56c-68c1e54e4e57 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:41:13.032262 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b66e47-c778-4d91-97f6-d8fea695a04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:41:13.032917 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b66e47-c778-4d91-97f6-d8fea695a04e admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:13.033202 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b66e47-c778-4d91-97f6-d8fea695a04e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:13.033478 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b66e47-c778-4d91-97f6-d8fea695a04e admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:13.034234 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b66e47-c778-4d91-97f6-d8fea695a04e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:41:13.035975 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b66e47-c778-4d91-97f6-d8fea695a04e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:41:13.045839 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1445/2884] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:41:13.049937 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36173494-1a08-436d-9897-f8544cc4b780 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:41:13.060608 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36173494-1a08-436d-9897-f8544cc4b780 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:41:13.065099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112d6ae2-2fa1-4f93-a59c-0be9e6e517ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:41:13.065992 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d6ae2-2fa1-4f93-a59c-0be9e6e517ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:13.066277 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d6ae2-2fa1-4f93-a59c-0be9e6e517ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:13.066547 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d6ae2-2fa1-4f93-a59c-0be9e6e517ed admin admin] PATH_INFO: `/v3/projects/e5e28b36dc2545d5b5ef95f599a6974b/users/9497d9a0ecba4ffca7b44b50a694b55f/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:13.081387 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112d6ae2-2fa1-4f93-a59c-0be9e6e517ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5e28b36dc2545d5b5ef95f599a6974b, user_id=9497d9a0ecba4ffca7b44b50a694b55f, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:41:13.084167 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112d6ae2-2fa1-4f93-a59c-0be9e6e517ed admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:41:13.097209 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1440/2885] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:41:13 2022] PUT /identity/v3/projects/e5e28b36dc2545d5b5ef95f599a6974b/users/9497d9a0ecba4ffca7b44b50a694b55f/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:41:13.107502 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2584e473-5e2d-4f1b-8d28-878101bd4d24 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:13.108021 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2584e473-5e2d-4f1b-8d28-878101bd4d24 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:13.108144 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2584e473-5e2d-4f1b-8d28-878101bd4d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:13.135296 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2584e473-5e2d-4f1b-8d28-878101bd4d24 None None] MFA Rules not processed for user `9497d9a0ecba4ffca7b44b50a694b55f`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:41:13.136689 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2584e473-5e2d-4f1b-8d28-878101bd4d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:41:13.203236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1446/2886] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:41:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:41:23.093269 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=30,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:41:23.200099 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=57,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:41:50.168894 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112d6ae2-2fa1-4f93-a59c-0be9e6e517ed admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:41:50.170600 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-112d6ae2-2fa1-4f93-a59c-0be9e6e517ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:41:50.229089 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112d6ae2-2fa1-4f93-a59c-0be9e6e517ed admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:41:50.233212 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acafd04-4e5b-46ba-a8bd-115208fa40a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:41:50.233908 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acafd04-4e5b-46ba-a8bd-115208fa40a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:50.234195 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acafd04-4e5b-46ba-a8bd-115208fa40a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:50.234474 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acafd04-4e5b-46ba-a8bd-115208fa40a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:50.244655 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acafd04-4e5b-46ba-a8bd-115208fa40a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:41:50.246166 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acafd04-4e5b-46ba-a8bd-115208fa40a7 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:41:50.255694 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1441/2887] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:41:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:41:50.280219 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2584e473-5e2d-4f1b-8d28-878101bd4d24 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:41:50.345918 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2584e473-5e2d-4f1b-8d28-878101bd4d24 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:41:50.351355 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96ba148-6828-4c5b-91aa-c188d7d569cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:41:50.352166 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ba148-6828-4c5b-91aa-c188d7d569cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:50.352461 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ba148-6828-4c5b-91aa-c188d7d569cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:50.352715 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ba148-6828-4c5b-91aa-c188d7d569cd admin admin] PATH_INFO: `/v3/users/9497d9a0ecba4ffca7b44b50a694b55f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:50.354005 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96ba148-6828-4c5b-91aa-c188d7d569cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9497d9a0ecba4ffca7b44b50a694b55f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:41:50.355979 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96ba148-6828-4c5b-91aa-c188d7d569cd admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:41:50.407613 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c96ba148-6828-4c5b-91aa-c188d7d569cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9497d9a0ecba4ffca7b44b50a694b55f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:41:50.408401 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c96ba148-6828-4c5b-91aa-c188d7d569cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9497d9a0ecba4ffca7b44b50a694b55f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:41:50.409063 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c96ba148-6828-4c5b-91aa-c188d7d569cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9497d9a0ecba4ffca7b44b50a694b55f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:41:50.416931 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c96ba148-6828-4c5b-91aa-c188d7d569cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9497d9a0ecba4ffca7b44b50a694b55f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:41:50.426070 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c96ba148-6828-4c5b-91aa-c188d7d569cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9497d9a0ecba4ffca7b44b50a694b55f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:41:50.444076 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1447/2888] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:41:50 2022] DELETE /identity/v3/users/9497d9a0ecba4ffca7b44b50a694b55f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:41:50.448578 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a437fb-e23f-4d43-876a-407567f2778f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:50.448920 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a437fb-e23f-4d43-876a-407567f2778f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:50.449230 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a437fb-e23f-4d43-876a-407567f2778f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:50.466992 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c4a437fb-e23f-4d43-876a-407567f2778f None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:41:50.468191 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c4a437fb-e23f-4d43-876a-407567f2778f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:41:50.528013 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1442/2889] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:41:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:41:50.538752 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96ba148-6828-4c5b-91aa-c188d7d569cd admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:41:50.540185 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c96ba148-6828-4c5b-91aa-c188d7d569cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:41:50.597807 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96ba148-6828-4c5b-91aa-c188d7d569cd admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:41:50.601524 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47766cb7-76d6-4499-bad0-99e22441b0e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:41:50.602143 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47766cb7-76d6-4499-bad0-99e22441b0e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:50.602429 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47766cb7-76d6-4499-bad0-99e22441b0e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:50.602708 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47766cb7-76d6-4499-bad0-99e22441b0e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:50.611744 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47766cb7-76d6-4499-bad0-99e22441b0e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:41:50.613199 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47766cb7-76d6-4499-bad0-99e22441b0e2 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:41:50.622848 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1448/2890] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:41:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:41:50.889921 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a437fb-e23f-4d43-876a-407567f2778f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:41:50.891353 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c4a437fb-e23f-4d43-876a-407567f2778f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:41:50.903261 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a437fb-e23f-4d43-876a-407567f2778f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:41:50.908514 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ad076-7301-4381-b124-d702208662a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:41:50.910027 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ad076-7301-4381-b124-d702208662a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:50.910316 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ad076-7301-4381-b124-d702208662a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:50.910643 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ad076-7301-4381-b124-d702208662a9 admin admin] PATH_INFO: `/v3/projects/e5e28b36dc2545d5b5ef95f599a6974b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:50.911847 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ad076-7301-4381-b124-d702208662a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5e28b36dc2545d5b5ef95f599a6974b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:41:50.913837 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ad076-7301-4381-b124-d702208662a9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:41:50.964100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d58ad076-7301-4381-b124-d702208662a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5e28b36dc2545d5b5ef95f599a6974b', 'request_id': 'req-d58ad076-7301-4381-b124-d702208662a9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:41:50.971601 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d58ad076-7301-4381-b124-d702208662a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5e28b36dc2545d5b5ef95f599a6974b', 'request_id': 'req-d58ad076-7301-4381-b124-d702208662a9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:41:50.978037 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d58ad076-7301-4381-b124-d702208662a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5e28b36dc2545d5b5ef95f599a6974b', 'request_id': 'req-d58ad076-7301-4381-b124-d702208662a9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:41:50.989758 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d58ad076-7301-4381-b124-d702208662a9 admin admin] The token cache is being invalidate because project e5e28b36dc2545d5b5ef95f599a6974b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:41:50.990043 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d58ad076-7301-4381-b124-d702208662a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:41:50.990927 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d58ad076-7301-4381-b124-d702208662a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:41:50.992460 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1443/2891] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:41:50 2022] DELETE /identity/v3/projects/e5e28b36dc2545d5b5ef95f599a6974b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:41:50.997389 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa55ab-775a-4af3-b0b1-b96612b05fc6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:50.997720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa55ab-775a-4af3-b0b1-b96612b05fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:50.998003 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa55ab-775a-4af3-b0b1-b96612b05fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:51.016586 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-caaa55ab-775a-4af3-b0b1-b96612b05fc6 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:41:51.017842 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-caaa55ab-775a-4af3-b0b1-b96612b05fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:41:51.034938 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:41:51.079974 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1449/2892] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:41:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:41:51.087808 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519c442-fc3d-45ac-bff9-eaeb05714239 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:51.088113 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519c442-fc3d-45ac-bff9-eaeb05714239 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:51.088407 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519c442-fc3d-45ac-bff9-eaeb05714239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:51.107634 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d519c442-fc3d-45ac-bff9-eaeb05714239 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:41:51.108854 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d519c442-fc3d-45ac-bff9-eaeb05714239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:41:51.117777 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1444/2893] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:41:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:41:51.122029 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa55ab-775a-4af3-b0b1-b96612b05fc6 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:41:51.132835 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa55ab-775a-4af3-b0b1-b96612b05fc6 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:41:51.138434 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a090c4-f658-4da5-b958-8b88f821416f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:41:51.139134 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a090c4-f658-4da5-b958-8b88f821416f admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:51.139464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a090c4-f658-4da5-b958-8b88f821416f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:51.139740 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a090c4-f658-4da5-b958-8b88f821416f admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:51.140534 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a090c4-f658-4da5-b958-8b88f821416f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 16:41:51.140816 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a090c4-f658-4da5-b958-8b88f821416f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:41:51.142678 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a090c4-f658-4da5-b958-8b88f821416f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:41:51.155186 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1450/2894] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 16:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 16:41:51.161181 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d519c442-fc3d-45ac-bff9-eaeb05714239 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:41:51.173168 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d519c442-fc3d-45ac-bff9-eaeb05714239 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:41:51.179848 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd41c5c-e3a9-4924-ae94-bf49deba04fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:41:51.180668 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd41c5c-e3a9-4924-ae94-bf49deba04fd admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:51.181043 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd41c5c-e3a9-4924-ae94-bf49deba04fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:51.181344 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd41c5c-e3a9-4924-ae94-bf49deba04fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:51.182155 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd41c5c-e3a9-4924-ae94-bf49deba04fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:41:51.184077 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd41c5c-e3a9-4924-ae94-bf49deba04fd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:41:51.204541 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:41:51.212970 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1445/2895] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:41:51 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:41:51.217127 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a090c4-f658-4da5-b958-8b88f821416f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:41:51.228029 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a090c4-f658-4da5-b958-8b88f821416f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:41:51.288013 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1dbcf-63aa-4590-b45a-1ed9d983122b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:41:51.288697 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1dbcf-63aa-4590-b45a-1ed9d983122b admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:51.288961 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1dbcf-63aa-4590-b45a-1ed9d983122b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:51.289236 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1dbcf-63aa-4590-b45a-1ed9d983122b admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:51.289989 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1dbcf-63aa-4590-b45a-1ed9d983122b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:41:51.291899 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1dbcf-63aa-4590-b45a-1ed9d983122b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:41:51.313327 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1451/2896] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:41:51 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:41:51.317518 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd41c5c-e3a9-4924-ae94-bf49deba04fd admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:41:51.328190 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd41c5c-e3a9-4924-ae94-bf49deba04fd admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:41:51.333383 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd30f8ae-6f4f-42c5-a26b-3ae8e1256873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:41:51.334095 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30f8ae-6f4f-42c5-a26b-3ae8e1256873 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:51.334379 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30f8ae-6f4f-42c5-a26b-3ae8e1256873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:51.334677 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30f8ae-6f4f-42c5-a26b-3ae8e1256873 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:51.335411 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd30f8ae-6f4f-42c5-a26b-3ae8e1256873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:41:51.337048 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd30f8ae-6f4f-42c5-a26b-3ae8e1256873 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:41:51.349662 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1446/2897] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:41:51.353812 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1dbcf-63aa-4590-b45a-1ed9d983122b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:41:51.364181 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1dbcf-63aa-4590-b45a-1ed9d983122b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:41:51.369301 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c5468-ab7a-43e1-bc16-83059402390d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:41:51.370010 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c5468-ab7a-43e1-bc16-83059402390d admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:51.370299 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c5468-ab7a-43e1-bc16-83059402390d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:51.370583 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c5468-ab7a-43e1-bc16-83059402390d admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:51.371325 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c5468-ab7a-43e1-bc16-83059402390d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:41:51.372926 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c5468-ab7a-43e1-bc16-83059402390d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:41:51.384268 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1452/2898] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:41:51.388405 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd30f8ae-6f4f-42c5-a26b-3ae8e1256873 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:41:51.399195 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd30f8ae-6f4f-42c5-a26b-3ae8e1256873 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:41:51.404092 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b37a7-6b59-4879-94b7-8e90607d4d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:41:51.404975 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b37a7-6b59-4879-94b7-8e90607d4d55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:51.405265 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b37a7-6b59-4879-94b7-8e90607d4d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:51.405545 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b37a7-6b59-4879-94b7-8e90607d4d55 admin admin] PATH_INFO: `/v3/projects/6ffeb9a20f074e62adfe22409c140083/users/0937ff8763ad4435ae2c631cc300a85e/roles/54195e660fc349978b39d5794765a095` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:51.420811 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b37a7-6b59-4879-94b7-8e90607d4d55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ffeb9a20f074e62adfe22409c140083, user_id=0937ff8763ad4435ae2c631cc300a85e, role_id=54195e660fc349978b39d5794765a095)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:41:51.423515 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b37a7-6b59-4879-94b7-8e90607d4d55 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:41:51.437339 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1447/2899] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:41:51 2022] PUT /identity/v3/projects/6ffeb9a20f074e62adfe22409c140083/users/0937ff8763ad4435ae2c631cc300a85e/roles/54195e660fc349978b39d5794765a095 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:41:51.442482 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785e34e-6c8c-4211-81b5-dd3ed3392015 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:51.442778 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785e34e-6c8c-4211-81b5-dd3ed3392015 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:51.443095 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785e34e-6c8c-4211-81b5-dd3ed3392015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:51.461381 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7785e34e-6c8c-4211-81b5-dd3ed3392015 None None] MFA Rules not processed for user `bf570352584f4974b4b425f3e760bf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:41:51.464071 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7785e34e-6c8c-4211-81b5-dd3ed3392015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:41:51.528210 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1453/2900] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:41:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:41:51.538494 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b37a7-6b59-4879-94b7-8e90607d4d55 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:41:51.540202 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-643b37a7-6b59-4879-94b7-8e90607d4d55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:41:51.598589 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b37a7-6b59-4879-94b7-8e90607d4d55 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:41:51.602272 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed721d-1ef2-4e22-99f7-0baaeafd56a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:41:51.602916 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed721d-1ef2-4e22-99f7-0baaeafd56a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:51.603198 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed721d-1ef2-4e22-99f7-0baaeafd56a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:51.603478 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed721d-1ef2-4e22-99f7-0baaeafd56a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:51.612927 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed721d-1ef2-4e22-99f7-0baaeafd56a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:41:51.614402 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed721d-1ef2-4e22-99f7-0baaeafd56a9 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:41:51.624534 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1448/2901] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:41:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 16:41:55.782373 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1cf45a-b602-4dc7-85c8-95cc1b638534 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:55.782898 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1cf45a-b602-4dc7-85c8-95cc1b638534 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:55.782971 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1cf45a-b602-4dc7-85c8-95cc1b638534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:55.821244 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0b1cf45a-b602-4dc7-85c8-95cc1b638534 None None] MFA Rules not processed for user `0937ff8763ad4435ae2c631cc300a85e`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:41:55.823038 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0b1cf45a-b602-4dc7-85c8-95cc1b638534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:41:55.850827 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:41:55.914218 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1454/2902] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:41:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:41:55.918848 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed721d-1ef2-4e22-99f7-0baaeafd56a9 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:41:55.928988 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed721d-1ef2-4e22-99f7-0baaeafd56a9 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:41:55.934213 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e1421-eaee-4562-8c62-4e82e03ea9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:41:55.935099 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e1421-eaee-4562-8c62-4e82e03ea9a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:55.935414 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e1421-eaee-4562-8c62-4e82e03ea9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:55.935685 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e1421-eaee-4562-8c62-4e82e03ea9a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:55.936418 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e1421-eaee-4562-8c62-4e82e03ea9a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:41:55.938430 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e1421-eaee-4562-8c62-4e82e03ea9a1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:41:55.966294 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1449/2903] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 16:41:55 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:41:55.970914 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1cf45a-b602-4dc7-85c8-95cc1b638534 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:41:55.982740 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1cf45a-b602-4dc7-85c8-95cc1b638534 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:41:56.045070 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c318a-d556-4793-8052-31c8503f996f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:41:56.045916 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c318a-d556-4793-8052-31c8503f996f admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:56.047397 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c318a-d556-4793-8052-31c8503f996f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:56.047714 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c318a-d556-4793-8052-31c8503f996f admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:56.048453 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c318a-d556-4793-8052-31c8503f996f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:41:56.050464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c318a-d556-4793-8052-31c8503f996f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:41:56.072466 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1455/2904] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 16:41:55 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 16:41:56.076666 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e1421-eaee-4562-8c62-4e82e03ea9a1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:41:56.088731 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e1421-eaee-4562-8c62-4e82e03ea9a1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:41:56.094457 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d25181-35dd-4313-beda-1d3ed0eb2aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:41:56.095125 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d25181-35dd-4313-beda-1d3ed0eb2aea admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:56.095504 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d25181-35dd-4313-beda-1d3ed0eb2aea admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:56.095860 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d25181-35dd-4313-beda-1d3ed0eb2aea admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:56.096728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d25181-35dd-4313-beda-1d3ed0eb2aea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:41:56.098939 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d25181-35dd-4313-beda-1d3ed0eb2aea admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:41:56.110986 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1450/2905] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:41:56.115161 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c318a-d556-4793-8052-31c8503f996f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:41:56.125960 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c318a-d556-4793-8052-31c8503f996f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:41:56.136020 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8760d86-ddb6-43b9-a02f-4c6e6da8537b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:41:56.136929 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8760d86-ddb6-43b9-a02f-4c6e6da8537b admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:56.137210 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8760d86-ddb6-43b9-a02f-4c6e6da8537b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:56.137498 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8760d86-ddb6-43b9-a02f-4c6e6da8537b admin admin] PATH_INFO: `/v3/projects/13166383be044973bea38697e3afc434/users/c39cc0b39b424643860301071a0f002e/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:56.153859 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8760d86-ddb6-43b9-a02f-4c6e6da8537b admin admin] RBAC: Authorizing `identity:create_grant(project_id=13166383be044973bea38697e3afc434, user_id=c39cc0b39b424643860301071a0f002e, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:41:56.156513 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8760d86-ddb6-43b9-a02f-4c6e6da8537b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:41:56.171098 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1456/2906] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:41:56 2022] PUT /identity/v3/projects/13166383be044973bea38697e3afc434/users/c39cc0b39b424643860301071a0f002e/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:41:56.175088 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d25181-35dd-4313-beda-1d3ed0eb2aea admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:41:56.186591 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d25181-35dd-4313-beda-1d3ed0eb2aea admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:41:56.246068 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01bf8b4-4c8e-4295-a4a2-a4c574aa0343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:41:56.246738 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01bf8b4-4c8e-4295-a4a2-a4c574aa0343 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:56.247021 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01bf8b4-4c8e-4295-a4a2-a4c574aa0343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:56.247293 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01bf8b4-4c8e-4295-a4a2-a4c574aa0343 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:56.248019 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01bf8b4-4c8e-4295-a4a2-a4c574aa0343 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:41:56.249638 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01bf8b4-4c8e-4295-a4a2-a4c574aa0343 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:41:56.260289 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1451/2907] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 16:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 16:41:56.264426 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8760d86-ddb6-43b9-a02f-4c6e6da8537b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:41:56.275417 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8760d86-ddb6-43b9-a02f-4c6e6da8537b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:41:56.279977 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870bee7a-3f9c-4c71-8579-9505b6143728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:41:56.280854 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870bee7a-3f9c-4c71-8579-9505b6143728 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:41:56.281145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870bee7a-3f9c-4c71-8579-9505b6143728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:41:56.281429 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870bee7a-3f9c-4c71-8579-9505b6143728 admin admin] PATH_INFO: `/v3/projects/13166383be044973bea38697e3afc434/users/c39cc0b39b424643860301071a0f002e/roles/475521b8a0174037b162f92022aea1a1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:41:56.283439 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870bee7a-3f9c-4c71-8579-9505b6143728 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13166383be044973bea38697e3afc434, user_id=c39cc0b39b424643860301071a0f002e, role_id=475521b8a0174037b162f92022aea1a1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:41:56.286117 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870bee7a-3f9c-4c71-8579-9505b6143728 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:41:56.296416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1457/2908] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 16:41:56 2022] PUT /identity/v3/projects/13166383be044973bea38697e3afc434/users/c39cc0b39b424643860301071a0f002e/roles/475521b8a0174037b162f92022aea1a1 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:42:01.122881 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6ee50-e056-4641-bdd8-b2883800b89a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:42:01.123378 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6ee50-e056-4641-bdd8-b2883800b89a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:42:01.123461 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6ee50-e056-4641-bdd8-b2883800b89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:42:01.162022 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-64d6ee50-e056-4641-bdd8-b2883800b89a None None] MFA Rules not processed for user `c39cc0b39b424643860301071a0f002e`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:42:01.163328 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-64d6ee50-e056-4641-bdd8-b2883800b89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:42:01.185479 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:42:01.259814 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1452/2909] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:42:01 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:42:01.272288 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870bee7a-3f9c-4c71-8579-9505b6143728 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:42:01.274222 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-870bee7a-3f9c-4c71-8579-9505b6143728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:42:01.357115 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870bee7a-3f9c-4c71-8579-9505b6143728 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:42:01.362748 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459d30f-9aa9-4967-bcd3-d2fa89a04536 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:42:01.363410 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459d30f-9aa9-4967-bcd3-d2fa89a04536 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:42:01.363698 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459d30f-9aa9-4967-bcd3-d2fa89a04536 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:42:01.363980 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459d30f-9aa9-4967-bcd3-d2fa89a04536 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:42:01.370864 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:42:01.426909 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6459d30f-9aa9-4967-bcd3-d2fa89a04536 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:42:01.428462 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6459d30f-9aa9-4967-bcd3-d2fa89a04536 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:42:01.438827 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1458/2910] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:42:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:42:02.442194 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257cd1e-fb18-436a-bcee-b61780a4a902 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:42:02.442754 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257cd1e-fb18-436a-bcee-b61780a4a902 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:42:02.442754 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257cd1e-fb18-436a-bcee-b61780a4a902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:42:02.468936 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b257cd1e-fb18-436a-bcee-b61780a4a902 None None] MFA Rules not processed for user `0937ff8763ad4435ae2c631cc300a85e`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:42:02.470204 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b257cd1e-fb18-436a-bcee-b61780a4a902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:42:02.478451 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1453/2911] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:42:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 16:42:11.433831 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=24 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:42:12.457694 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=24 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:43:05.970707 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459d30f-9aa9-4967-bcd3-d2fa89a04536 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:43:05.972640 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6459d30f-9aa9-4967-bcd3-d2fa89a04536 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:43:06.084931 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459d30f-9aa9-4967-bcd3-d2fa89a04536 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:43:06.090352 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7153a47-516e-47fc-a34e-030925259c4c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822d9d3521db4a9688f51c6313c7dd74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:43:06.091145 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7153a47-516e-47fc-a34e-030925259c4c service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:43:06.091486 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7153a47-516e-47fc-a34e-030925259c4c service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:43:06.091775 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7153a47-516e-47fc-a34e-030925259c4c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:43:06.097260 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f7153a47-516e-47fc-a34e-030925259c4c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:43:06.109691 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7153a47-516e-47fc-a34e-030925259c4c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:43:06.111382 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7153a47-516e-47fc-a34e-030925259c4c service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:43:06.123552 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1459/2912] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:43:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:43:09.327300 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b257cd1e-fb18-436a-bcee-b61780a4a902 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:43:09.409939 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b257cd1e-fb18-436a-bcee-b61780a4a902 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:43:09.414575 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa364cf4-8e9c-4610-bd85-3629b515fbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:43:09.415618 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa364cf4-8e9c-4610-bd85-3629b515fbe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:43:09.415904 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa364cf4-8e9c-4610-bd85-3629b515fbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:43:09.416190 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa364cf4-8e9c-4610-bd85-3629b515fbe4 admin admin] PATH_INFO: `/v3/users/0937ff8763ad4435ae2c631cc300a85e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:43:09.417395 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa364cf4-8e9c-4610-bd85-3629b515fbe4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0937ff8763ad4435ae2c631cc300a85e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:43:09.419381 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa364cf4-8e9c-4610-bd85-3629b515fbe4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:43:09.468219 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fa364cf4-8e9c-4610-bd85-3629b515fbe4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0937ff8763ad4435ae2c631cc300a85e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:43:09.471196 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fa364cf4-8e9c-4610-bd85-3629b515fbe4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0937ff8763ad4435ae2c631cc300a85e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:43:09.471852 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fa364cf4-8e9c-4610-bd85-3629b515fbe4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0937ff8763ad4435ae2c631cc300a85e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:43:09.479100 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fa364cf4-8e9c-4610-bd85-3629b515fbe4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0937ff8763ad4435ae2c631cc300a85e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:43:09.487010 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fa364cf4-8e9c-4610-bd85-3629b515fbe4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0937ff8763ad4435ae2c631cc300a85e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:43:09.506871 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1454/2913] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:43:09 2022] DELETE /identity/v3/users/0937ff8763ad4435ae2c631cc300a85e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:43:09.682894 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7153a47-516e-47fc-a34e-030925259c4c service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:43:09.713393 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7153a47-516e-47fc-a34e-030925259c4c service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:43:09.770688 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe17f5-3f86-4c61-b5b0-8f82c48f9cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:43:09.771553 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe17f5-3f86-4c61-b5b0-8f82c48f9cc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:43:09.771839 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe17f5-3f86-4c61-b5b0-8f82c48f9cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:43:09.772122 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe17f5-3f86-4c61-b5b0-8f82c48f9cc0 admin admin] PATH_INFO: `/v3/projects/6ffeb9a20f074e62adfe22409c140083` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:43:09.773321 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fe17f5-3f86-4c61-b5b0-8f82c48f9cc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ffeb9a20f074e62adfe22409c140083)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:43:09.775257 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fe17f5-3f86-4c61-b5b0-8f82c48f9cc0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:43:09.826017 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f0fe17f5-3f86-4c61-b5b0-8f82c48f9cc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ffeb9a20f074e62adfe22409c140083', 'request_id': 'req-f0fe17f5-3f86-4c61-b5b0-8f82c48f9cc0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:43:09.832463 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f0fe17f5-3f86-4c61-b5b0-8f82c48f9cc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ffeb9a20f074e62adfe22409c140083', 'request_id': 'req-f0fe17f5-3f86-4c61-b5b0-8f82c48f9cc0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:43:09.839869 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f0fe17f5-3f86-4c61-b5b0-8f82c48f9cc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ffeb9a20f074e62adfe22409c140083', 'request_id': 'req-f0fe17f5-3f86-4c61-b5b0-8f82c48f9cc0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:43:09.850049 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f0fe17f5-3f86-4c61-b5b0-8f82c48f9cc0 admin admin] The token cache is being invalidate because project 6ffeb9a20f074e62adfe22409c140083 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:43:09.850345 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f0fe17f5-3f86-4c61-b5b0-8f82c48f9cc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:43:09.850976 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f0fe17f5-3f86-4c61-b5b0-8f82c48f9cc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:43:09.852522 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1460/2914] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:43:09 2022] DELETE /identity/v3/projects/6ffeb9a20f074e62adfe22409c140083 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:43:09.856672 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa364cf4-8e9c-4610-bd85-3629b515fbe4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:43:09.858087 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fa364cf4-8e9c-4610-bd85-3629b515fbe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:43:09.882359 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa364cf4-8e9c-4610-bd85-3629b515fbe4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:43:09.929171 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:43:09.940390 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e523328e-aec4-4ed7-a5df-2d0d47e430e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:43:09.941284 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523328e-aec4-4ed7-a5df-2d0d47e430e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:43:09.941567 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523328e-aec4-4ed7-a5df-2d0d47e430e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:43:09.941883 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523328e-aec4-4ed7-a5df-2d0d47e430e8 admin admin] PATH_INFO: `/v3/users/c39cc0b39b424643860301071a0f002e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:43:09.943094 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e523328e-aec4-4ed7-a5df-2d0d47e430e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c39cc0b39b424643860301071a0f002e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:43:09.945052 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e523328e-aec4-4ed7-a5df-2d0d47e430e8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:43:09.991430 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e523328e-aec4-4ed7-a5df-2d0d47e430e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c39cc0b39b424643860301071a0f002e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:43:09.992090 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e523328e-aec4-4ed7-a5df-2d0d47e430e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c39cc0b39b424643860301071a0f002e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:43:09.992728 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e523328e-aec4-4ed7-a5df-2d0d47e430e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c39cc0b39b424643860301071a0f002e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:43:09.999602 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e523328e-aec4-4ed7-a5df-2d0d47e430e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c39cc0b39b424643860301071a0f002e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:43:10.006253 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e523328e-aec4-4ed7-a5df-2d0d47e430e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c39cc0b39b424643860301071a0f002e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:43:10.025256 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1455/2915] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:43:09 2022] DELETE /identity/v3/users/c39cc0b39b424643860301071a0f002e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:43:10.202633 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe17f5-3f86-4c61-b5b0-8f82c48f9cc0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:43:10.204359 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f0fe17f5-3f86-4c61-b5b0-8f82c48f9cc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:43:10.213323 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:43:10.222734 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe17f5-3f86-4c61-b5b0-8f82c48f9cc0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:43:10.294877 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2e9bc-8b43-46e0-b4bb-4c85fce30158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:43:10.295863 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2e9bc-8b43-46e0-b4bb-4c85fce30158 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:43:10.296166 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2e9bc-8b43-46e0-b4bb-4c85fce30158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:43:10.296448 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2e9bc-8b43-46e0-b4bb-4c85fce30158 admin admin] PATH_INFO: `/v3/projects/13166383be044973bea38697e3afc434` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:43:10.298519 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f2e9bc-8b43-46e0-b4bb-4c85fce30158 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13166383be044973bea38697e3afc434)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:43:10.300464 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f2e9bc-8b43-46e0-b4bb-4c85fce30158 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:43:10.346674 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-29f2e9bc-8b43-46e0-b4bb-4c85fce30158 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13166383be044973bea38697e3afc434', 'request_id': 'req-29f2e9bc-8b43-46e0-b4bb-4c85fce30158'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:43:10.353315 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-29f2e9bc-8b43-46e0-b4bb-4c85fce30158 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13166383be044973bea38697e3afc434', 'request_id': 'req-29f2e9bc-8b43-46e0-b4bb-4c85fce30158'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:43:10.362062 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-29f2e9bc-8b43-46e0-b4bb-4c85fce30158 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13166383be044973bea38697e3afc434', 'request_id': 'req-29f2e9bc-8b43-46e0-b4bb-4c85fce30158'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:43:10.372837 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-29f2e9bc-8b43-46e0-b4bb-4c85fce30158 admin admin] The token cache is being invalidate because project 13166383be044973bea38697e3afc434 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:43:10.373131 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-29f2e9bc-8b43-46e0-b4bb-4c85fce30158 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:43:10.373801 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-29f2e9bc-8b43-46e0-b4bb-4c85fce30158 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:43:10.375416 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1461/2916] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:43:10 2022] DELETE /identity/v3/projects/13166383be044973bea38697e3afc434 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:43:11.337817 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e523328e-aec4-4ed7-a5df-2d0d47e430e8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:43:11.339304 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e523328e-aec4-4ed7-a5df-2d0d47e430e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:43:11.416764 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e523328e-aec4-4ed7-a5df-2d0d47e430e8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:43:11.465909 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa91ac-5955-4a22-aef6-04b86c71d790 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87c0598af7714aeba08fffbb6805a49e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:43:11.466560 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa91ac-5955-4a22-aef6-04b86c71d790 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:43:11.466846 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa91ac-5955-4a22-aef6-04b86c71d790 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:43:11.467153 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa91ac-5955-4a22-aef6-04b86c71d790 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:43:11.470064 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8dfa91ac-5955-4a22-aef6-04b86c71d790 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:43:11.498450 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:43:11.533851 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfa91ac-5955-4a22-aef6-04b86c71d790 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:43:11.535303 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfa91ac-5955-4a22-aef6-04b86c71d790 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:43:11.544725 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1456/2917] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:43:11 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:43:20.369731 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=50,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:43:21.540283 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=15 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:43:56.792214 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2e9bc-8b43-46e0-b4bb-4c85fce30158 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:43:56.793668 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-29f2e9bc-8b43-46e0-b4bb-4c85fce30158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:43:56.880271 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2e9bc-8b43-46e0-b4bb-4c85fce30158 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:43:56.885373 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b15afb-5c80-49ef-adc7-25acb354636f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822d9d3521db4a9688f51c6313c7dd74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:43:56.886150 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b15afb-5c80-49ef-adc7-25acb354636f service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:43:56.886480 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b15afb-5c80-49ef-adc7-25acb354636f service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:43:56.886826 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b15afb-5c80-49ef-adc7-25acb354636f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:43:56.888932 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c8b15afb-5c80-49ef-adc7-25acb354636f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:43:56.901629 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b15afb-5c80-49ef-adc7-25acb354636f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:43:56.903227 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b15afb-5c80-49ef-adc7-25acb354636f service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:43:56.914346 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1462/2918] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:43:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:44:06.910124 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=39 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:44:43.177231 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7df25-8f88-4976-bc4c-e955ea4b02de None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:44:43.177810 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7df25-8f88-4976-bc4c-e955ea4b02de None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:44:43.177810 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7df25-8f88-4976-bc4c-e955ea4b02de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:44:43.224786 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6ca7df25-8f88-4976-bc4c-e955ea4b02de None None] MFA Rules not processed for user `c967025cedf94cc5a69ed5818a0fbd11`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 16:44:43.226030 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6ca7df25-8f88-4976-bc4c-e955ea4b02de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:44:43.293016 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1457/2919] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 16:44:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 16:44:43.526283 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b15afb-5c80-49ef-adc7-25acb354636f service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:44:43.527703 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c8b15afb-5c80-49ef-adc7-25acb354636f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:44:43.586606 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b15afb-5c80-49ef-adc7-25acb354636f service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:44:43.590221 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54e0dd8-58fb-440e-9e69-b692ae929f31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:44:43.590870 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e0dd8-58fb-440e-9e69-b692ae929f31 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:44:43.591163 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e0dd8-58fb-440e-9e69-b692ae929f31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:44:43.591446 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e0dd8-58fb-440e-9e69-b692ae929f31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:44:43.593210 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e54e0dd8-58fb-440e-9e69-b692ae929f31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:44:43.603372 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54e0dd8-58fb-440e-9e69-b692ae929f31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:44:43.604806 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54e0dd8-58fb-440e-9e69-b692ae929f31 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:44:43.613244 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1463/2920] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:44:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:44:43.935251 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7df25-8f88-4976-bc4c-e955ea4b02de None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:44:43.936695 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6ca7df25-8f88-4976-bc4c-e955ea4b02de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:44:43.956510 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7df25-8f88-4976-bc4c-e955ea4b02de None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:44:43.963460 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1709e66-cfb4-4f08-8c1a-28a8490051c9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822d9d3521db4a9688f51c6313c7dd74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:44:43.964266 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1709e66-cfb4-4f08-8c1a-28a8490051c9 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:44:43.964631 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1709e66-cfb4-4f08-8c1a-28a8490051c9 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:44:43.964980 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1709e66-cfb4-4f08-8c1a-28a8490051c9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:44:43.967376 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d1709e66-cfb4-4f08-8c1a-28a8490051c9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:44:43.983193 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1709e66-cfb4-4f08-8c1a-28a8490051c9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:44:43.985065 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1709e66-cfb4-4f08-8c1a-28a8490051c9 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:44:43.997141 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1458/2921] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:44:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:44:45.648982 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54e0dd8-58fb-440e-9e69-b692ae929f31 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:44:45.657813 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54e0dd8-58fb-440e-9e69-b692ae929f31 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:44:45.661462 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50d197-1082-4920-b42a-3bba179982dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:44:45.662162 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50d197-1082-4920-b42a-3bba179982dd service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:44:45.662489 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50d197-1082-4920-b42a-3bba179982dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:44:45.662741 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50d197-1082-4920-b42a-3bba179982dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:44:45.664619 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6d50d197-1082-4920-b42a-3bba179982dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:44:45.675410 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50d197-1082-4920-b42a-3bba179982dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:44:45.676851 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50d197-1082-4920-b42a-3bba179982dd service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:44:45.685620 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1464/2922] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:44:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 16:44:46.399608 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1709e66-cfb4-4f08-8c1a-28a8490051c9 service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:44:46.401111 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d1709e66-cfb4-4f08-8c1a-28a8490051c9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:44:46.477888 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1709e66-cfb4-4f08-8c1a-28a8490051c9 service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:44:46.483337 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afcb197-5c34-43d2-be44-10d6ae589675 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:44:46.484598 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcb197-5c34-43d2-be44-10d6ae589675 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:44:46.484890 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcb197-5c34-43d2-be44-10d6ae589675 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:44:46.485170 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcb197-5c34-43d2-be44-10d6ae589675 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:44:46.487020 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1afcb197-5c34-43d2-be44-10d6ae589675 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:44:46.501135 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afcb197-5c34-43d2-be44-10d6ae589675 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:44:46.502768 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afcb197-5c34-43d2-be44-10d6ae589675 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:44:46.513791 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1459/2923] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:44:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:44:49.987294 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50d197-1082-4920-b42a-3bba179982dd service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:44:50.001648 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50d197-1082-4920-b42a-3bba179982dd service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:44:50.007238 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec454c-ca0d-4733-aee3-5f2186a78997 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:44:50.007899 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec454c-ca0d-4733-aee3-5f2186a78997 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:44:50.008410 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec454c-ca0d-4733-aee3-5f2186a78997 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:44:50.008518 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec454c-ca0d-4733-aee3-5f2186a78997 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:44:50.010348 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f4ec454c-ca0d-4733-aee3-5f2186a78997 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:44:50.022996 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec454c-ca0d-4733-aee3-5f2186a78997 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:44:50.024603 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec454c-ca0d-4733-aee3-5f2186a78997 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:44:50.034662 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1465/2924] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:44:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:44:51.793271 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afcb197-5c34-43d2-be44-10d6ae589675 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:44:51.805243 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afcb197-5c34-43d2-be44-10d6ae589675 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:44:51.811202 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31030425-631c-402b-9586-5a8d2ab9883d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '092553a86f8848dba8c7271f5c0b2c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:44:51.811860 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31030425-631c-402b-9586-5a8d2ab9883d service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:44:51.812145 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31030425-631c-402b-9586-5a8d2ab9883d service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:44:51.812427 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31030425-631c-402b-9586-5a8d2ab9883d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:44:51.814222 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-31030425-631c-402b-9586-5a8d2ab9883d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:44:51.827519 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31030425-631c-402b-9586-5a8d2ab9883d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:44:51.829175 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31030425-631c-402b-9586-5a8d2ab9883d service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:44:51.841117 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1460/2925] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:44:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:45:00.030386 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=38 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:45:01.836068 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=87 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:45:41.394316 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec454c-ca0d-4733-aee3-5f2186a78997 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:45:41.405272 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec454c-ca0d-4733-aee3-5f2186a78997 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:45:41.409793 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85037070-2562-4f24-9bae-ad5a7448c650 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:45:41.410444 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85037070-2562-4f24-9bae-ad5a7448c650 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:45:41.410731 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85037070-2562-4f24-9bae-ad5a7448c650 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:45:41.411014 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85037070-2562-4f24-9bae-ad5a7448c650 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:45:41.412792 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-85037070-2562-4f24-9bae-ad5a7448c650 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:45:41.479229 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85037070-2562-4f24-9bae-ad5a7448c650 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:45:41.480691 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85037070-2562-4f24-9bae-ad5a7448c650 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:45:41.490989 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1466/2926] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:45:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 16:45:42.021256 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31030425-631c-402b-9586-5a8d2ab9883d service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:45:42.022744 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-31030425-631c-402b-9586-5a8d2ab9883d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:45:42.036672 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31030425-631c-402b-9586-5a8d2ab9883d service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:45:42.041855 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a02c5-0446-46da-906e-5bb99b037c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:45:42.042729 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a02c5-0446-46da-906e-5bb99b037c3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:45:42.043010 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a02c5-0446-46da-906e-5bb99b037c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:45:42.043283 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a02c5-0446-46da-906e-5bb99b037c3e admin admin] PATH_INFO: `/v3/users/c967025cedf94cc5a69ed5818a0fbd11` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:45:42.044506 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a02c5-0446-46da-906e-5bb99b037c3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c967025cedf94cc5a69ed5818a0fbd11)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:45:42.046531 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a02c5-0446-46da-906e-5bb99b037c3e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:45:42.098122 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8b7a02c5-0446-46da-906e-5bb99b037c3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c967025cedf94cc5a69ed5818a0fbd11'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:45:42.098854 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8b7a02c5-0446-46da-906e-5bb99b037c3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c967025cedf94cc5a69ed5818a0fbd11'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:45:42.099499 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8b7a02c5-0446-46da-906e-5bb99b037c3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c967025cedf94cc5a69ed5818a0fbd11'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:45:42.108208 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8b7a02c5-0446-46da-906e-5bb99b037c3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c967025cedf94cc5a69ed5818a0fbd11'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:45:42.116442 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8b7a02c5-0446-46da-906e-5bb99b037c3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c967025cedf94cc5a69ed5818a0fbd11'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:45:42.136656 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1461/2927] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:45:42 2022] DELETE /identity/v3/users/c967025cedf94cc5a69ed5818a0fbd11 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:45:42.547893 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85037070-2562-4f24-9bae-ad5a7448c650 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:45:42.560492 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85037070-2562-4f24-9bae-ad5a7448c650 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:45:42.624687 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266489b-eae0-4148-ba22-c826ab30c882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:45:42.625553 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266489b-eae0-4148-ba22-c826ab30c882 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:45:42.625878 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266489b-eae0-4148-ba22-c826ab30c882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:45:42.626167 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266489b-eae0-4148-ba22-c826ab30c882 admin admin] PATH_INFO: `/v3/projects/64e1c0ba9480420188d60da173aa1e45` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:45:42.627386 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5266489b-eae0-4148-ba22-c826ab30c882 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64e1c0ba9480420188d60da173aa1e45)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:45:42.629305 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5266489b-eae0-4148-ba22-c826ab30c882 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:45:42.678117 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5266489b-eae0-4148-ba22-c826ab30c882 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64e1c0ba9480420188d60da173aa1e45', 'request_id': 'req-5266489b-eae0-4148-ba22-c826ab30c882'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:45:42.685289 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5266489b-eae0-4148-ba22-c826ab30c882 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64e1c0ba9480420188d60da173aa1e45', 'request_id': 'req-5266489b-eae0-4148-ba22-c826ab30c882'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:45:42.693417 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5266489b-eae0-4148-ba22-c826ab30c882 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64e1c0ba9480420188d60da173aa1e45', 'request_id': 'req-5266489b-eae0-4148-ba22-c826ab30c882'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:45:42.703949 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5266489b-eae0-4148-ba22-c826ab30c882 admin admin] The token cache is being invalidate because project 64e1c0ba9480420188d60da173aa1e45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:45:42.704250 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5266489b-eae0-4148-ba22-c826ab30c882 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:45:42.705720 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5266489b-eae0-4148-ba22-c826ab30c882 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:45:42.707274 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1467/2928] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:45:42 2022] DELETE /identity/v3/projects/64e1c0ba9480420188d60da173aa1e45 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:45:52.132565 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:45:52.700779 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=81,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:47:50.533765 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a02c5-0446-46da-906e-5bb99b037c3e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:47:50.535295 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8b7a02c5-0446-46da-906e-5bb99b037c3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:47:50.615076 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a02c5-0446-46da-906e-5bb99b037c3e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:47:50.667001 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad04a9e1-f01d-46d2-86c5-691d56cb9672 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1654fdf5ba6b46ba8d9b86a83a76a4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:47:50.667708 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04a9e1-f01d-46d2-86c5-691d56cb9672 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:47:50.668272 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04a9e1-f01d-46d2-86c5-691d56cb9672 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:47:50.668484 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04a9e1-f01d-46d2-86c5-691d56cb9672 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:47:50.670547 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ad04a9e1-f01d-46d2-86c5-691d56cb9672 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:47:50.757164 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad04a9e1-f01d-46d2-86c5-691d56cb9672 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:47:50.758691 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad04a9e1-f01d-46d2-86c5-691d56cb9672 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:47:50.777181 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1462/2929] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:47:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:47:53.845040 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266489b-eae0-4148-ba22-c826ab30c882 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:47:53.846509 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5266489b-eae0-4148-ba22-c826ab30c882 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:47:53.927968 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266489b-eae0-4148-ba22-c826ab30c882 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:47:53.933070 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83230c88-158d-4dca-85cd-a6e1b1c953e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822d9d3521db4a9688f51c6313c7dd74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:47:53.933757 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83230c88-158d-4dca-85cd-a6e1b1c953e6 service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:47:53.934043 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83230c88-158d-4dca-85cd-a6e1b1c953e6 service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:47:53.934324 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83230c88-158d-4dca-85cd-a6e1b1c953e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:47:53.936229 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-83230c88-158d-4dca-85cd-a6e1b1c953e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:47:53.948107 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83230c88-158d-4dca-85cd-a6e1b1c953e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:47:53.949766 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83230c88-158d-4dca-85cd-a6e1b1c953e6 service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:47:53.959905 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1468/2930] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:47:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:47:56.846779 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad04a9e1-f01d-46d2-86c5-691d56cb9672 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:47:56.848511 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ad04a9e1-f01d-46d2-86c5-691d56cb9672 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:47:56.861548 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad04a9e1-f01d-46d2-86c5-691d56cb9672 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:47:56.867329 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2618b-8a87-40a4-83af-84f82b68e22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:47:56.868243 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2618b-8a87-40a4-83af-84f82b68e22c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:47:56.868531 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2618b-8a87-40a4-83af-84f82b68e22c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:47:56.868805 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2618b-8a87-40a4-83af-84f82b68e22c admin admin] PATH_INFO: `/v3/users/762f5eba8cbe4bb9b40062e6cd1e9929` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:47:56.870071 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde2618b-8a87-40a4-83af-84f82b68e22c admin admin] RBAC: Authorizing `identity:delete_user(user_id=762f5eba8cbe4bb9b40062e6cd1e9929)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:47:56.872030 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde2618b-8a87-40a4-83af-84f82b68e22c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:47:56.887664 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:47:56.930591 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fde2618b-8a87-40a4-83af-84f82b68e22c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '762f5eba8cbe4bb9b40062e6cd1e9929'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:47:56.931280 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fde2618b-8a87-40a4-83af-84f82b68e22c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '762f5eba8cbe4bb9b40062e6cd1e9929'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:47:56.931900 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fde2618b-8a87-40a4-83af-84f82b68e22c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '762f5eba8cbe4bb9b40062e6cd1e9929'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:47:56.940450 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fde2618b-8a87-40a4-83af-84f82b68e22c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '762f5eba8cbe4bb9b40062e6cd1e9929'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:47:56.950043 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fde2618b-8a87-40a4-83af-84f82b68e22c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '762f5eba8cbe4bb9b40062e6cd1e9929'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:47:56.970223 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1463/2931] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:47:56 2022] DELETE /identity/v3/users/762f5eba8cbe4bb9b40062e6cd1e9929 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:47:57.146727 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83230c88-158d-4dca-85cd-a6e1b1c953e6 service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:47:57.160409 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83230c88-158d-4dca-85cd-a6e1b1c953e6 service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:47:57.220668 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08ff10-69ef-4072-b4a1-66c1de7d605e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:47:57.221531 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08ff10-69ef-4072-b4a1-66c1de7d605e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:47:57.221836 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08ff10-69ef-4072-b4a1-66c1de7d605e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:47:57.222113 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08ff10-69ef-4072-b4a1-66c1de7d605e admin admin] PATH_INFO: `/v3/projects/9d55db285dae4de79e1ecc442129bab5` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:47:57.223289 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e08ff10-69ef-4072-b4a1-66c1de7d605e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d55db285dae4de79e1ecc442129bab5)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:47:57.225178 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e08ff10-69ef-4072-b4a1-66c1de7d605e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:47:57.272080 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4e08ff10-69ef-4072-b4a1-66c1de7d605e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d55db285dae4de79e1ecc442129bab5', 'request_id': 'req-4e08ff10-69ef-4072-b4a1-66c1de7d605e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:47:57.278688 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4e08ff10-69ef-4072-b4a1-66c1de7d605e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d55db285dae4de79e1ecc442129bab5', 'request_id': 'req-4e08ff10-69ef-4072-b4a1-66c1de7d605e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:47:57.285408 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4e08ff10-69ef-4072-b4a1-66c1de7d605e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d55db285dae4de79e1ecc442129bab5', 'request_id': 'req-4e08ff10-69ef-4072-b4a1-66c1de7d605e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:47:57.297655 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4e08ff10-69ef-4072-b4a1-66c1de7d605e admin admin] The token cache is being invalidate because project 9d55db285dae4de79e1ecc442129bab5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:47:57.297955 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4e08ff10-69ef-4072-b4a1-66c1de7d605e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:47:57.298899 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4e08ff10-69ef-4072-b4a1-66c1de7d605e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:47:57.300263 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1469/2932] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:47:57 2022] DELETE /identity/v3/projects/9d55db285dae4de79e1ecc442129bab5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:47:57.304926 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2618b-8a87-40a4-83af-84f82b68e22c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:47:57.306519 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fde2618b-8a87-40a4-83af-84f82b68e22c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:47:57.321319 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2618b-8a87-40a4-83af-84f82b68e22c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:47:57.381900 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba0a36e-751c-47b2-80f8-8311ac3109c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:47:57.382784 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0a36e-751c-47b2-80f8-8311ac3109c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:47:57.383070 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0a36e-751c-47b2-80f8-8311ac3109c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:47:57.383377 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0a36e-751c-47b2-80f8-8311ac3109c8 admin admin] PATH_INFO: `/v3/users/ceeae25b81044757a915c36b3cdd1b1e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:47:57.518017 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba0a36e-751c-47b2-80f8-8311ac3109c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ceeae25b81044757a915c36b3cdd1b1e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:47:57.519957 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba0a36e-751c-47b2-80f8-8311ac3109c8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:47:57.564835 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0ba0a36e-751c-47b2-80f8-8311ac3109c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ceeae25b81044757a915c36b3cdd1b1e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:47:57.565427 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0ba0a36e-751c-47b2-80f8-8311ac3109c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ceeae25b81044757a915c36b3cdd1b1e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:47:57.566065 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0ba0a36e-751c-47b2-80f8-8311ac3109c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ceeae25b81044757a915c36b3cdd1b1e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:47:57.573055 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0ba0a36e-751c-47b2-80f8-8311ac3109c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ceeae25b81044757a915c36b3cdd1b1e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:47:57.579710 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0ba0a36e-751c-47b2-80f8-8311ac3109c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ceeae25b81044757a915c36b3cdd1b1e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:47:57.592545 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:47:57.598815 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1464/2933] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 16:47:57 2022] DELETE /identity/v3/users/ceeae25b81044757a915c36b3cdd1b1e => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:47:57.768935 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08ff10-69ef-4072-b4a1-66c1de7d605e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:47:57.770469 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4e08ff10-69ef-4072-b4a1-66c1de7d605e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:47:57.789606 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08ff10-69ef-4072-b4a1-66c1de7d605e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:47:57.810353 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:47:57.851688 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd1209-c2a4-43dd-bb50-e33f757bd6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf570352584f4974b4b425f3e760bf0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2cb644b2d994629b4b82065db94ebac', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:47:57.852501 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd1209-c2a4-43dd-bb50-e33f757bd6e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:47:57.852789 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd1209-c2a4-43dd-bb50-e33f757bd6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:47:57.853066 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd1209-c2a4-43dd-bb50-e33f757bd6e3 admin admin] PATH_INFO: `/v3/projects/5fd0067e651f464db47b7c377b40ac34` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:47:57.864029 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dd1209-c2a4-43dd-bb50-e33f757bd6e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fd0067e651f464db47b7c377b40ac34)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:47:57.866284 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dd1209-c2a4-43dd-bb50-e33f757bd6e3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:47:57.912220 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-44dd1209-c2a4-43dd-bb50-e33f757bd6e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fd0067e651f464db47b7c377b40ac34', 'request_id': 'req-44dd1209-c2a4-43dd-bb50-e33f757bd6e3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:47:57.920365 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-44dd1209-c2a4-43dd-bb50-e33f757bd6e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fd0067e651f464db47b7c377b40ac34', 'request_id': 'req-44dd1209-c2a4-43dd-bb50-e33f757bd6e3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:47:57.928105 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-44dd1209-c2a4-43dd-bb50-e33f757bd6e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fd0067e651f464db47b7c377b40ac34', 'request_id': 'req-44dd1209-c2a4-43dd-bb50-e33f757bd6e3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:47:57.938455 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-44dd1209-c2a4-43dd-bb50-e33f757bd6e3 admin admin] The token cache is being invalidate because project 5fd0067e651f464db47b7c377b40ac34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 16:47:57.938747 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-44dd1209-c2a4-43dd-bb50-e33f757bd6e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:47:57.939366 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-44dd1209-c2a4-43dd-bb50-e33f757bd6e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 16:47:57.940888 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1470/2934] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 16:47:57 2022] DELETE /identity/v3/projects/5fd0067e651f464db47b7c377b40ac34 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 16:48:07.602127 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:48:07.935582 dv-cinder-850668-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 16:48:59.697400 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba0a36e-751c-47b2-80f8-8311ac3109c8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 16:48:59.698816 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0ba0a36e-751c-47b2-80f8-8311ac3109c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:48:59.785599 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba0a36e-751c-47b2-80f8-8311ac3109c8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 16:48:59.844419 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bbd19c-0481-4369-b120-f296c52b2ab6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822d9d3521db4a9688f51c6313c7dd74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1ac00308714c39a79aae09fbb73021', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 16:48:59.845064 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbd19c-0481-4369-b120-f296c52b2ab6 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 16:48:59.847792 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbd19c-0481-4369-b120-f296c52b2ab6 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 16:48:59.848072 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbd19c-0481-4369-b120-f296c52b2ab6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 16:48:59.849924 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-43bbd19c-0481-4369-b120-f296c52b2ab6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 16:48:59.861876 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bbd19c-0481-4369-b120-f296c52b2ab6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 16:48:59.863454 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bbd19c-0481-4369-b120-f296c52b2ab6 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 16:48:59.873733 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1465/2935] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 16:48:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 16:49:09.869489 dv-cinder-850668-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=63 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}